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5 



RELATED APPLICATIONS 



This application claims priority ftom U.S. Provisional Application 60/196314, filed April 12, 
2000, to Columbus et al., which is incorporated herein by reference. 



The present inventicn relates generally to analyzing performance of investment professionals, 
and more particularly, to a system, method, and con^uter readable medium storing computer- 
executable instructions for evaluating the performance of securities analysts, investment analysts and/or 
15 other investment related individuals and/or entities. 



20 always been considered a prudent exercise. These analysts possess expertise in evaluating investments, 
such as securities, mutual fimds, bonds, and the like. Most specialize in specific industries or sectors to 
allow for more in-depth research- During the course of providing investment advice, each atialyst may 
make any number of upward and/or downward forecast revisions on securities in their area of expertise. 
These forecast revisions, or sinoply revisions, are changes in opinion on how much a company is likely 

25 to earn per share. 

When these analysts speak, institutional as well as individual investors listen, driving stock 
prices sharply up or down d^>ending on the nature of the analysts' latest pronouncenaents. As can be 
ttnagined, some analysts are noore influential than others. For instance, some analysts, affecticHiately 

30 named * axes' , are so accurate that their conoments carry disproportionate weight While none are 

infallible, it is important to at least know and consider what professional analysts have to say about a 
particular stock of interest With access to this information, users can make their own assessments and 
invest accordingly. Accordingly, we have determined that a need exists for efficiendy disseminating 
such analyst information to the public. A need also exists for a system, method, and computer readable 

35 medium containing instructions utilizable for providing not only information concerning or considering 
an analyst's revisions, but also information pertaining to the reliability and/or accuracy of the revisions. 



10 



TECHNICAL FIELP 



BACKGROUND ART 



In the context of securities and other types of investments, obtaining the advice of analysts has 
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Several prior art techniques have not adequately addressed these needs. For example, U.S. 
Patent 5,132,899 to Fox discloses a stock and cash portfolio development system. As depicted in prior 
ait FIG. 1 of the present invention (FIG. 1 of Fox) , the system of Fox uses data gathering and 
processing methodology to produce a system where a list of stocks and a cash position is generated and 
5 purchased for investment and operating accounts. Specifically, the system integrates three areas of 
data: investment performance for investment managers; Federal Securities and Exchange Commission 
reports filed quarterly by investment managers; and financial characteristics for a number of stocks, to 
produce a stock portfolio. For example, in box 1, investment managers are screened to produce a 
subset for analysis. From there, managers are selected (box 2) and sorted into descending order simply 
10 according to rates of return on investments (box 3). Subsequently, a consultant reviews the results (box 
4) and prints the top five mangers (box 5). 

Similarly, U.S. Patent 4,566,066 to Towers relates to a securities valuation systenx As shown 
in pricM- art FIG. 2 of the present application (FIG. 1 of Towers) , the system of Towers , comprised of 
components 10-23. produces securities portfolio valuation schedules for multiple simultaneous users. 
15 In Towers , a customer communicates with the system through terminal 10 to access and edit accounts 
in user file 12, By using a CUSIP routine 18 and an AMOUNT routine to reflect stock splits and 
dividends since the last stock pricing, the system of Towers produces and displays 23 a selected stock 
portfolio valuation. 

20 In U.S. Patent 5,8 12,987 to T.nslrin at ah. an investment fund management system manages 

^sets in one or more investment fimds over a specified period of time. This system, comprised of 
conq>onents 31-39, determines a strategic investment mix of assets in a particular fund periodically as a 
function of changing risk. Each fund is managed by manipulating the investment mix of the fund in 
accordance with criteria related to a diminishing length of time to a horizon where cash will be 

25 withdrawn thercfronL In prior art FIG. 3 of the present invention (HG- 6 of Luskin etal.> , the 

investment mix is adjusted by first obtaining investor portfolio infannatic«i 35 and market data 36. This 
data is used to forecast market risks and returns 37, and to determine the portfolio risk 38. Then, the 
anticipated cash flow stream 39, the discount function 34, and present value of future cash flow 33 are 
calculated before producing an optimized portfolio 32 of assets. 

30 

In U.S. Patent 5,761,442 to Bair et al. , a data processing system selects securities and 
constructs an investment portfolio based on a set of artificial neural networks (prior art FIG. 4 of the 
present application, FIG. 2 of BarretaL). The system comprises conqxments 41-70 and is designed to 
model and track the performance of each security in a given capital market and output a parameter 
35 which is related to the expected risk adjusted return for the security. Each artificial neural network is 
trained using a number of fundanoental and price and volume history input parameters 10, 20, 30 about 
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the security and the underlying index. The system combines the expected return/appreciation potential 
data 50 for each security via an optimizadon process 60 to construct an investment portfolio which 
satisGes aggregate statistics. The data processing system receives input from the capital market and 
periodicaDy evaluates the performance of the mvestment portfolio, rebalancing whenever necessary to 
correct performance degradations 70. 

However while many of these prior art references disclose adequate methods of managing 
investment funds and portfolios, none make any mention of evaluating an analyst's or other investment 
trader's perfomoance. Hence, what is lacking in the prior art is a technique directed not only toward an 
investnoent fund, but rather a technique for evaluating analysts and other traders or investment 
professionals/advisors based on their influence and accuracy. Accordingly, a need exists for a system, 
method, and compute readable mediimi containing instrucdons utilizable for not only disseminating 
informadon concerning an analyst's or trader's actions, but also fcx evaluating the analyst's cm* trader's 
perfomoance. 

In line with the above, a need exists for a system, method, and computer readable medium 
containing instrucdons utilizable not only fc^ providing raw information and data but also for 
evaluating performance based on returns observed after a pardcular revision, historical consistency, 
and/or the number of revisions made by the analyst or investing endty. 

Furthermore, a need exists for a system, method, and computer readable a)edium containing 
instructions utilizable for considering performance based on revisions including a change in opinion 
including an upward or downward change in opinion of the analyst with regard to at least one 
investmenL 

A need also exists for a system, method, and con^)uter readable medium containing instrucdons 
utilizable for adjusting a performance measure to accentuate a nnmber of revisions issued by an analyst 

Yet another need exists for a system, method, and computer readable medium containing 
instrucdons utilizable for adjusting a perfomoance measure according to a return amount adjustment 
accentuating the measure according to a return amount realized on an investment 

Still another need exists for a system, method, and computer readable medium containing 
instructions utilizable for adjusting a perfOTmance measure according to a small return penalty 
penalizing an analyst for smaU returns. 
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SUMMARY OF THE INVENTION 
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It is a feature and an advantage of the present invention to provide a system, method, and 
computer readable medium containing instructions utilizable for not only disseminating information 
concOTiing an analyst's or trader's actions, but also for evaluating the analyst's or trader's performance. 

It is another feature and advantage of the present invendon to provide a system, method, and 
conaputer readable medium containing instructions utilizable not only for providing raw information 
and data but also for evaluating performance based on returns observed after a particular revision, 
historical consistency, and/or the number of revisions made by the analyst or investing entity. 

It is another feature and advantage of the present invention to provide a system, method, and 
computer readable medium containing instructions utilizable for considering performance based on 
revisions including a change in opinion including an upward or downward change in opinion of the 
analyst with regard to at least one investment 

It is another feature and advantage of the present invention to provide a system, method, and 
computer readable medium containing instructions utilizable for adjusting a performance measure to 
accentuate a nimiber of revisions issued by an analyst 

It is another feature and advantage of the present iavention to provide a system, method, and 
coiiq)uter readable medium containing instructions utilizable for adjusting a performance measure 
acccxxiing to a return amount adjustment accentuating the measure according to a return amount 
realized on an investment 

It is another feature and advantage of the present invention to provide a system, method, and 
computer readable n^dium containing instructions utilizable for adjusting a performance measure 
according to a small return penalty penalizing an analyst for small returns. 

One embodiment of the present invention, particularly applicable to securities analysts, is now 
summarized. 
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System/Processing Description 

Using a proprietary calculation, analysts may be given a series of performance scores that rank them 
relative to (1) other analysts in the "universe" and (2) other analysts that make predictions about a 
5 specific company. A total of five sccffes may be produced for each analyst: 

Analyst Bny Score - Represents the analyst's Buy Score for all companies he/she covers, relative to the 
Universe of Analysts 

Analyst Sell Score - Represents the analysts Sell Score for all conq)anies he/she covers, relative to the 
10 Universe of Analysts 

Analyst Company Buy Score - Represents the analyst's Score for a specific Company 
Analyst Company Sell Score - Represents the analyst's Score for a specific Conq>any 
Analyst OveraO Score - Represents the analysts overall Score for all conq>anies that he/she covers 

15 All scores will be calculated based on an event - the revision of an analyst's earnings forecast 

Data Preparation Steps 

The following nmst be complete before the Analyst calculations are begun: 

20 1. Analyst Backend [ABE] -this process supplies the data firom any suitable data base for the 
calculations. 

2. ABE data relating to Brokers may be removed fit)m the dB - in this embodiment we do not 
want to calculate Analyst scares for these non-analyst entities 

25 Analyst Scoring Logic 

The following logic will be used to calculate the sccxes for all analyst/company combinations in the 
Analyst database with at least 2 earnings forecast revisions that are at least 60 days (3 calendar months) 
old. These raw scores will later be ranked to fonnulate the final Analyst scores: 

30 Raw Buy Analyst Score - Note, all inputs are based on calculated values related to earnings estimate 
revision changes 

For every analyst_id and company_id combination in the analyst_con5)any_total table with 2 or more 
earnings estimate revisions^ calculate a raw scare using the following logic (the logic is first 
35 summarized and then explained in detail): 
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Step 1 Calculate the average 5 day, 20 day and 60 day returns for earnings revisions [5/20/60 event 
returns] for each analyst/company combination (done by ABE) 

Step 2 Calculate the t -statistic [t-stat] values for 5/20/60 event returns in absolute terms 
Step 3 Calculate the degrees of fteedom [DF] values for 5/20/60 event returns 
5 Step 4 Look up probability value using DF and [t stat] for 5/20/60 event returns 
Step 5 Calculate raw score using conditional logic 
Step 6 Calculate adjustment to raw score 
Step 7 Calculate adjusted raw score 

Step 8 Calculate second adjustment to raw score and new raw score 
10 Step 9 Eliminate raw scores with returns greater than 1000% 

Step 10 Sort and rank the raw scores to-produce the final Analyst Scores. 



For every analyst_id and company„id combination in the analyst„company_total table in the Analyst 
dB with analyst_company_totaL events >= 2, perform the foUowing logic: 

15 



Stepl 

The 5 day, 20 day and 60 day return values are calculated by ABE for each analyst/company 
combination: 

20 

Average 5day return = analyst_coinpany_total.avg_5dy_retum_buy 
Average 20day return = analyst_company_total.avg_20dy_retum_buy 
Average 60day return =aiialyst_con5)any_total.avg_60dy_retum_buy 

25 Step 2 

The t-stat needs to be calculated in absolute terms (Le., a non-negative number) for the 5 day, 20 day 
and 60 day returns using the foUowing formulas: 

The 5/20/60day_abs_tstaX values may be rounded to 2 decimal places 

30 

Before beginning this calculation, we must ensure that we will not be dividing by zero. If the standard 
deviation is zero, then we will set the final raw score to zero and move to the next analyst/conopany 
combinadon. 



35 



IF analyst_company„total-sd_5dy_buy = 0, THEN raw_score_buy = 0 
And start over with the next analyst/company buy conobination 
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IF analyst_company_total.sd_2{)dy_buy = 0, THEN raw_score_buy = 0 
And start over with the next analyst/company buy combination 

5 IF analyst_coinpany_total.sd_60dy_buy = 0, THEN raw„scOTe_buy = 0 
And start over with the next analyst/company buy combination 



Sday _abs _tstai _buy = 



analyst ^company _totaljiyg _5dy ^return _buy 



analyst ^company _total^d _Sdy _buy 
^ ^Janafyst _ company _ total aium _ 5dy _ e vent _ buy 



10 TSiday _abs _tstat _buy = 



analyst _ company _totalMvg _ 20dFy _retum _^buy 



analyst ^company _total.sd _ ZOrfy _buy ^ 
^ ^analyst ^company _totaljium _2J(kly ^event _buy ^ 



60day _abs _tstat _buy 



analyst ^company _totaljivg _60dy _retum _buy 



^ analyst ^company _ totaled _ 60dy _ buy ^ 
^analyst ^company _totaljtum _ 60dy _event _buy 



15 



20 



Steps 



The degrees of fireedom are calculated by subtracting 1 from the number of observations for each return 
value: 

5day _ dfjbuy - analyst_company_totaLnum_5dy_eveni_buy'l 
20day_dfjbuy = analyst_coT7tpany_JotaLmm_20dy_event_buy'l 



60day_dfjbuy = analyst joompanyJotaLman_60dy_ev€ntJ)iiy'l 



25 Step 4 
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For each return period, lcx)k up the probability using the absolute t-stat and DF for each respective 
calcuiatioa as inputs to a lookiq) table: 

IF 5/20/60day^df_buy > 100, THEN set 5/2Q/60day_df.buy = 100 

5 

IF 5/20/60_abs_tstat_buy > 20, THEN set 5/20^60day_abs_tstat_buy = 20 
IF 5/20/60day_abs tstat_buy < .01, THEN set 5/2a^60day_jTOb_buy = 0 
10 5day_prob_buy = lookup value indicated by using 5day_abs tstat_buy and 5day„df_buy 

20day_prob_buy = lookup value indicated by using 20day_abs_tstat_buy and 20day_df_buy 
60day_prob jMiy = lookup value indicated by using 60day_abs_tstat_buy and 60day_dt_buy 

15 

Steps 

The conditional raw score may now be calculated. The conditional raw scare will be based on the 
average of the conditional raw scores for the 5 day period, 20 day period and for the 60 day period, 
which will then be multiplied by 100. The condition to determine whether the probability is added or 
20 subtracted is based on whether the average return is positive or negative, the formula is as follows 
( cond _ raw _ score „ buy )' 

[1 - (5 -H or - Sday _ prob _ buy)] -H [1 - (J + or - IXkUiy _ prob _ buy)] + [1 - (.5 + or - €Oday _ prob _ buy)} 

3 

25 Logic: 

5 day portion of calculation 
IF analyst_company_total.avg_5dy_retum_buy > 0 
THEN subtract Sday _prob_buy from .5 in numerator 

30 

- This next case means that we have a negative average return for the events and we may add the 
probability 

ELSE add 5day__prob_buy to .5 in numerator 

35 




wo 01/80124 



PCT/US01/n69l 



9 

20day portion of calculation 

IF aiialyst_coinpany_total_avg_20dy_retiini buy > 0 

THEN subtract 20day_prob__buy firom .5 in nunaerator 

5 - This next case means that we have a negative average return for the events and we may add the 
probability 



ELSE add 2Ckiay_j)rob_buy to .5 in nuroerator 



10 60day portion o/calculation 

EF analyst_coEq)any_total.avg_60dy_retum„buy > 0 
THEN subtract 60day_4>rob_buy firom .5 in numerator 



- This next case means that we have a negative average return for the events and we may add the 
15 probability 

ELSE add 60day^rob_buy to .5 in numeratCH- 

Final step 

20 Take the average of the three calculated values above and multiply the result by 100 



Step 6 

Calculate the adjustment to the conditional raw score using the following formula (the adjustment is 
generally negative): 



25 



adj _ raw _ bity - (-20) * 



1 



fnum_5dy_ev€n^s_buy•^tvm^_2(kty_e^^nts_buy + num_60dy_evenis_buy 

f 3 



- num_5dy_events_buy, num 20dy_events_buy and num_20dy_events_buy are taken from the 
analyst company_total table— 

30 

Note, the constant above may be changed for Analysts. 
Step? 

Calculate the raw score by adding the adjustment to the conditional raw score: 
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raw_score_buy = (cond_raw„score_buy) + (adj_raw_buy) 

— here we are adding a negative number (adj_raw_seD), therefore the adjustnaent should generally be 
5 negative- 

Stepg 

new_raw_score_buy = raw_scoTe_buy + (adj2_ raw_buy) 
10 Calculate second adjustment to raw_score_buy, variable aanoe: adj2_raw_buy 
The adj2 ^w_buy is conqjrised of the following 4 components: 

(1) 5 day returns portion of calculation 

15 IF analyst_conipany_total.avg_5dy_retum_buy > 10 
THEN 5day_adj2_buy =10 

— This next case means that we have a 5 day return less or equal than 1000% and should use the 
following fonnula. Note the formula is raised to the third power which is equivalent to the cube root — 

20 

ELSE 5day_adj2_buy = (100 * analyst_company_total.avg_5dy_retuni_buy)^^ 

(2) 20 day returns portion of calculation 

IF analyst_company_total.avg_20dy_retum_buy > 10 
25 THEN 20day_adj2_buy = 10 

— This next case means that we have a 20 day return less than or equal 1000% and should use the 
following formula. Note the formula is raised to the third power which is equivalent to the cube root — 

30 ELSE 20day_ adj2_buy = (100 * analyst„company_total.avg_20dy_retum_buy)^^ 

(3) 60 day returns portion of calculation 

IF analyst„company_total.avg_60dyretum _buy > 10 
THEN 60day_adj2_buy =10 

35 - This next case means that we have a 60 day return less than or equal 1000% and should use the 

following fmnula. Note the formula is raised to the third power which is equivalent to the cube root - 
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1/3 

ELSE 60day_ adj2_buy = (100* analyst_coiiipany_total.avg_60dy_retuni_buy) 
(4) Small Returns penalty 

IF analyst„coinpany_total.avg_60dy_retuni_buy >0.15 
5 THEN small_Tet_buy_adj = 0 

— This next case means that we positive returns less than or equal 15%, so we will impose a penalty on 
the raw_buy_score — 

ELSE small_ret_buy_adj = (100* (0.15 - analyst_company_total.avg_60dy_retum_buy)) 
Final step 

10 Combine the 4 components using the following formula to arrive at the final adj_raw: 

adj2_raw_buy = (5day_buy_adj2 / 3) + (20day_buy.adj2 / 3) + (60day_buy_adj2 / 3> 
small„ret_buy_adj 

Step9 

Final step is to elinounate returns from the final scoring that have 60 day returns greater than 1000% 

15 IF anaIyst_conq)any_total-avg 60dy_retum_buy > 10 
THEN filter this raw score in the final scoring process 

Step 10 

Analyst Scoring Logic - Ranked (Final) Score 

The following ranking system will be used to determine the final scores for each of the analyst scores: 

20 Important Note: Only analyst that cover companies within the Analyst Universe should be given a fijial 
Analyst score in this example. 

Analyst Boy Score 

Once all of the raw scores have beed calculated, we can then rank the raw buy scores by sorting the 
results in descending order and separate this list into 100 groups. The group with the highest set of 
25 scores will be given the ranking of 100 (signifying the best performance), and the group with the lowest 
set of scores will be given the ranking of 1 (signifying the worst). The analysts in each group should 
then be assigned a score equal to the group that they are in. 

Step 1 Develop list of analysts who have received a raw_buy„score at conq>anies within the 

Analyst Universe 

30 Step 2 Count the number of analysts in the list created in Step 1 
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Step 3 Sort raw^scoie list descending for aU analyst id/con]pany_id where there is a score. 

[Note: If a secondary sort is needed, use analyst company_totaLavg.60day return descending]. 
Step 4 Use the look up table below to divide the ranked list into seven groups, and then within 

each group divide the ranked raw-score list into an equal number of observations for each score within 
5 the group (as defined in Table A shown below). 

Step 5 Record the score for the associated analystjd and company.id combinations in the group 

Raw SeB Analyst Score - Note, all inputs are based on calculated values related to earnings estimate 
revision changes 

10 

For every analyst id and conq)any-id combination in the analyst conq>any total table with 2 or more 
earnings estimate revisions, calculate a raw score using the following logic (the logic is first 
summarized and then explained in detail): 



15 


Step 1 


Calculate the average 6 day, 20 day and 60 day returns for earnings revisicms [5/20/60 




event returns] for each analyst/company combination (done by ABE) 




Step 2 


Calculate the t -statistic [t-stat] values for 5/20/60 event returns in absolute terms 




Step 3 


Calculate the degrees of freedom EDF] values for 5/20/60 event returns 




Step 4 


Look up probability value using DF and [t-stat] for 5/20/60 event returns 


20 


Step 5 


Calculate raw score using conditional logic 




Step 6 


Calculate adjustment to raw score 




Step? 


Calculate adjusted raw score 




Step 8 


Calculate second adjustment to raw score and new raw score 




Step 9 


Eliminate raw scores with returns greater than 1(X)0% 


25 


Step 10 


Sort and rank the raw scores to produce the final Analyst Sell Scores. 



In this example, for every analyst id and company_id combination in the analyst_conq>any_total table 
in the Analyst dB with analyst_company_total.events >= 2 perfonn the following logic: 

30 Step 1 

The 5 day, 20 day and 60 day return values are calculated by ABE for each analyst/conq>any 
combination. 

Average 5day return = analyst_company_total.avg_5dy_retunusell 
35 Average 20day return = analyst_conipany_total.avg„20dy_retum_sell 
Average 60day return = analyst„company_total.avg_60dy_retum_sell 
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Step 2 

The t-slat needs to be calculated in absolute terms (Le., a non-negative number) for the 5 day, 20 day 
and 60 day returns using the following fc^mulas: 

5 

The 5/20/60day_abs_tstat values may be rounded to 2 decimal places 



Before beginning this calculatioa, we must ensure that we wiU not be dividing by zero. If the standard 
deviation is zero, then we will set the final raw score to zero and move to the next analyst/company 
10 combination. 



IF anaIyst„company_total.sd_5dy_seIls = 0, THEN raw_scorc = 0 
And start over with the next analyst/company buy combination 



15 IF analyst_conipany_total.sd_20dy„sells = 0, THEN raw„scorc sell = 0 
And start over with the next analyst/company buy combination 



20 



EF anaiyst_company_total.sd_60dy„sells = 0, THEN raw_score_sell = 0 
And start over with the next analyst/company buy combination 



Sday _ab5 _t5tat _ sell = 



analyst _ company _ total Jtvg _ 5dy _ return _ sell 



analyst ^company _total^d _ 5dy _ seU 
^analyst _ company total Jium _ Sdy _ event _ seU 



TXiday _abs _tstat __seU = 



analyst _ company _ total javg _ 20dfy „ return _ sell 



analyst _ company _ total ^d _ 20rfy _ sell 

^ ^analyst ^company _ total Jtum _ 20dy _ event _ sell 



25 60day _abs ^tstat _sell = 



analyst ^company _totaluivg_60(fy _retum_seU 



^ analyst ^company _ total ^d _60dy _ sell ^ 
^Janalyst _ company _ total Jium _ 6(kfy _ event _ sell 
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Step 3 

The degrees of fireedom are simply calculated by subCractmg 1 from the number of observations for 
each return value: 

5 5day_df_sell =aiiaIyst_company~total.num_5dy_event_eveni_sell-l 
20day_df_S€ll = analyst jcompanyjtotaLnum_20dy_event_sell' I 
60day_£^_sell = aiuilyst_company_totaLnun_6(kty_eveTU_seU-l 
Step 4 

For each return period, lookup the probability using the absolute t stat and DF for each respective 
10 calculation as inputs to a lookup table: 

IF 5/20/60day„df_seU > 100, THEN set 5/20/60day_di_seU = 100 

IF 5/20/60 abs_tstat_sell > 20. THEN set 5/20/60day_abs_tstat_seU = 20 

15 

IF 5/20/60day_abs_tstat seU < .01. THEN set 5/2Q/60dayj?rDb_seU = 0 
5day_prob„sell = lookup value indicated by using 5day_abs_tstat_sell and 5day_df_sell 
20 20day_prob_sell = lookup value indicated by using 20day_abs_tstat_seU and 20day_df_sell 
60day_prob_sell = lookup value indicated by using 60day_abs_tstat_sell and 60day_df_sell 
Steps 

25 The conditional raw score may now be calculated. The conditional raw score will be based on the 
average of the conditional raw scores for the 5 day period, 20 day period and for the 60 day period, 
which will then be multiplied by 100. The condition to detemrine whether the probability is added or 
subtracted is based on whethCT the average return is positive or negative, the formula is as follows 
( cond _ raw _ score _ sell ): 

P - (.5 + or - 5day _ prob^sell)] + [1 - (.5 -t- or - 20day _ prob _ sell)] + [1 - (-5 + or - 60day_ prob_seUy\ 




Logic: (Note: We are now using less than zero, which is opposite of the Buy logic) 
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5 day portion of calculation 

IF analysCcompany total.avg_5dy_rctum_sell < 0 

THEN subtract 5day_prob l&rom .5 in numerate^' 

5 

- This next case means that we have a positive average return for the events and we may add the 
probability 

ELSE add 5day_prob to .5 in numerator 

10 

20day portion of calculation 

IF analyst coinpany_total.avg_20dy_retum_sell < 0 

THEN subtract 20day_prob from .5 in numerator 

15 - This next case means that we have a positive average return for the events and we need to add the 
probability 

ELSE add 20day_prob to .5 in nunoerator 

20 60day portion of calculation 

IF analyst_conipany_total.avg_60dy_retum sell < 0 
THEN subtract 60day j>rob from .5 in numerator 

- This next case means that we have a positive average return for the events and we need to add the 
25 probability 

ELSE add GQAzyjptoh to .5 in nmnerator 

Fimd step 

30 Take the average of the three calculated values above and multiply the result by 100 
Step 6 

Calculate the adjustment to the conditional raw score using the following formula (the adjustment will 
always be negative): 

35 
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1 



adj _raw_5ea= {-20)* 




num _ Srfy , events _ sell + num _ lOdy _ events _ seU + num _ 60dy _ events _ sell 



3 



J 



- nuin_5dy_events sell, num_20dy„events_sell and num_60dy_events_sell are all taken firom the 
analyst_coinpany„total table- 

Step7 

Calculate the raw sell score by adding the adjustment to the conditional raw score: 
raw_score_sell = (cond_raw„score_sell) + (adj_raw_sell) 

- here we are adding a negative number (adj_raw_sellX therefore the adjustment should generally be 
negative — 

Step 8 

new_raw__score_selI = raw_score_sell + (adj2_jraw_seU) 

Calculate second adjustment to raw_score_sell, variable name: adj2_raw„sell 

The adj2 raw is comprised of the following 4 components: 

(1) 5 day returns portion of calculation 

IF analyst company_total.avg_5dy_retum_sell > 10 

THEN 5day_seil_adj2 = 10 

- This next case means that we have a 5 day return less or equal than 1000% and should use the 
following formula. Note the fOTmula is raised to the third power which is equivalent to the cube root - 

ELSE 5day_sell_adj2 = (100 * analyst_company„total.avg_5dy_retum_sell)^ 



(2) 20 day returns portion of calculation 

EF analyst_conipany_total^vg_20dy_retuin_sell > 10 

THEN 20day_seU_adj2 = 10 
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- This next case means that we have a 20 day return less than ch* equal 1000% and should use the 
foUowing formula. Note the fonnula is raised to the third power which is equivalent to the cube root - 

ELSE 20day_sell_a£lj2 = (100 * analyst_conipany_total.avg_20dy_retuni_seil)^ 

5 

(3)60 day returns portion of calculation 

IF analyst_company_total.avg_60dy_retuni_sell > 10 

THEN 60day_seILadj2 = 10 

10 - This next case nieans that we have a 60 day return less than or equal 1000% and should use the 

following formula. Note the fonnula is raised to the third power which is equivalent to the cube root - 

ELSE 60day_sell_adj2 = (100 * analyst_conipany_totaLavg_60dy_retum_sell)^'^ 

1 5 (a) Small Returris penalty 

IF analyst_corDpany_total.avg_60dy_retum_sell < -0.08 
THEN small_ret„sell_adj = 0 

- This next case means that we negate returns less than or equal -8%, so we will iiiq)ose a penalty on the 
20 raw _seU_score - 

ELSE small_ret_sell_adj = (100 * (-0.08 - analyst_conq)any_total.avg_60dy_retuin_sell)) 

Final step 

25 Combine the 4 conq)onents using the following formula to arrive at the final adj2__raw_sell: 

adj2_raw_sell = (5day_seU.adj2/3) + (20day_sell_adj2/3) + (60day„adj2/3) + small_ret_sell_adj 
Step 9 

30 Hnal step is to eliminate returns from the final scoring that have 60 day returns greater than 1000% 

IF analyst_con3pany_total.avg_60dy_retum_sell > 10 
THEN filter this raw score in the final scoring process 



35 



Step 10 

Analyst Scoring Logic - Ranked (Final) Score 
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The following rankiiig system may be used to determine the fibaal scores for each of the analyst scores: 

iDoportant Note: Only analysts that cover companies within the Analyst Universe should be given a 
final Analyst score in this example 

Analyst Sell Score 

Once all of the raw sell scores have been calculated, we can then rank the raw sell scores by sorting the 
results in descending order and separate this list into 100 groups. The group with the highest set of 
scores will be given the ranking of 100 (signifying the best sell performance), and the group with the 
lowest set of scores will be given the ranking of 1 (signifying the worst sell performance). The analysts 
in each group should then be assigned a score equal to the group that they are in. 

Step 1 Develop list of analysts who have received a raw_sell_score at companies within the Analyst 
Universe 

15 Step 2 Count the number of analysts in the list created in Step 1 

Step 3 ScMt raw_sell_score list descending for all analyst_id/company_id wh^e there is a score, 
[Note: If a secondary sort is needed, use analyst conipany_total.avg_60dy_retum_sell descending]. 
Step 4 Use the look up table shown below to divide the ranked first into seven groups, and then 
within each group divide the ranked raw.sell score list mto an equal number of observations for each 

20 score within the group (as defined in the table shown below). 

Step 5 Record the score for the associated analyst Jd and company Jd combinations in the group 



LookUp Table A 

25 

This look up table provides the information needed to assign the Analyst Buy Score and Analyst Sell 
Score for each analyst 

Using the rankings determined in Step 3 of Analyst Buy Score/Analyst Sell Score procedures derive the 
30 Analyst Buy Score and Analyst Sell Score split the rankings into the 7 groups in the table below. 



Group 


Percentages of Total 


Analyst Buy Score 


Analyst Buy Score 




Observations 


Calculation 
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1 


Top 5.0% (rounded 
up to the Dearest 
whole number) 


Divide the top 5% of 
total observations into 
10 groups with an 
equal number of 
observations in each 
group 


91 to 100 


2 


Next 12.5% (i.e. read 
sorted records until 
reach 17.5% of the 
total number o 
individuals from Step 
1 of the scoring 
procedures) 


Divide the 12.5% of 
total observations into 
15 groups with an 
equal number of 
observations in each 
group 


76 to 90 


3 


Next 18.0% (Le. read 
sorted records until 
reach 35.5% of the 
total number of 
individuals from Step 
1 of the scoring 
procedures) 


Divide the 18.0% of 
total observations into 
15 groups with an 
equal number of 
observations in each 
group 


61 to 75 


4 


Next 29.0% (i.e. read 
sorted reccsxis until 
reach 64.5% of die 
total number of 
individuals from Step 
1 of the scoring 
procedures) 


Divide the 29.0% of 
total observations into 
20 groups with an 
equal number of 
observations in each 
group 


41 to 60 


5 


Next 18.0% (Le. read 
sorted records until 
reach 82.5% of the 
total number of 
individuals from Step 
1 of the scoring 
procedures) 


Divide die 18.0% of 
total observations into 
15 groups with an 
equal number of 
observations in each 
group 


26 to 40 
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6 


Next 12.5% (i.e. read 


Divide d>e 12.5% of 


11 to 25 




sorted records until 


total observations into 






reach 95.0% of the 


15 groups with an 






total number of 


equal number of 






individuals firom Step 


observations in each 






1 of the scoring 


group 






procedures) 






7 


Last 5.0% (rounded 


Divide the last 5% of 


1 to 10 




up to the nearest 


total observations into 






whole number) (i.e. 


10 groups with an 






read sorted records 


equal number of 






until reachJOO.0% of 


observations in each 






the total nuntiber of 


group 






individuals from Step 








1 of the scoring 








procedures) 







In accordance with the principles of the present invention, an analyst's performance (or other 
investment professional or investment advisor) is evaluated by utilizing information pertaining to at 
least one revision made by the analyst involving at least one investment This evaluation includes 
5 determining a conditional performance score indicative of the analyst's performance relative to other 
analysts. The performance sccwe is determined, at least in part, by considering an average historical 
performance of the investment following the revision. In addition, the perfcHmance score is also 
determined using a historical consistency of the analyst's performances with respect to revisions or 
reports involving the investment, and a number of revisions, recommendations, or opinions made by the 
10 analyst Then, the perfc^mance score may be adjusted according to one or more adjustments to 
generate a final performance score. 

These performance scores nCiay be determined based on analyst recommendations, such as 
upward or downward revisions or a combination of both. In particular, a score based on upward 

15 revisions, G€ an upward revision performance score, corresponds to the predictive nature of an analyst 
with respect to predicting or influencing increases in earning. In contrast, a score based on downward 
revisions, or a downward revision performance score, corresponds to the predictive nature of an analyst 
with respect to predicting or influencing decreases in earnings. Similarly, a combined score or, for 
example, an average of the two previous scores, corresponds to the overall predictive nature of the 

20 analyst 
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Hence, the present inventioa provides a method^ system, and computer readable medium 
storing con5)uter executable instructions for evaluating an analyst's performance by utilizing 
information pertaining to at least one revision made by the analyst involving at least one investment In 
5 one embodiment, the present invention includes determining a conditional performance score indicative 
of the analyst's performance relative to other analysts. This conditional performance score is 
determined at least in part by considering an average historical performance of the at least one 
investment following the at least one revision, a historical consistency of the analyst's perforniances 
with respect to revisions involving the at least one investment, and a nimiber of revisions made by the 
10 analyst, hi addition, the present invention includes adjusting the conditional performance score of the 
analyst according to a return amount adjustment to generate a final performance score. 

In another embodiment, the present invention provides a method, system, and con^)uter 
readable medium storing compute exectitable instructions for evaluating an analyst's performance. 

15 This embodiment includes utilizing information pertaining to at least one revision made by the analyst 
involving at least one investment, including an upward or downward change in opinion of the analyst 
with regard to at least one investment In addition, the present invention includes calculating a raw * 
conditional performance score indicative of the analyst's perfcMrmance relative to other analysts, said 
raw ccxiditional performance score determined at least in part by considering a measure of variability of 

20 the analyst's performance, an average historical performance of the at least one investment following 
the at least one revision, a standard deviation of the at least one revision, a number of revisions made by 
the analyst, and a likelihood that the at least one revision will actually produce an expected result. 
From there, the invention includes adjusting said raw conditional performance score according to a 
retum amount adjustment to produce a final performance score. 

25 

In another embodiment, the present invention provides a method of evaluating an analyst's 
perfomaance by utilizing information pertaining to at least one revision made by the analyst involving at 
least one investment. In this embodiment, the invention includes calculating a raw conditional 
performance score indicative of the analyst's performance relative to othCT analysts. The raw 

30 conditional performance score may be determined at least in part by considering a measure of 
variability of the analyst's performance, an average historical performance of the at least one 
investment following the at least one revision, a standard deviation of the at least one revision, a 
number of revisions made by the analyst, and a likelihood that the at least one revision will actually 
produce an expected result Subsequently, the invention adjusts the raw conditional performance score 

35 according to a noodifier accentuating the number of revisions made by the analyst In addition, the raw 
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conditional perfonnance score naay be adjusted according to a small return penalty penalizing the 
analyst for grnall returns to produce a final performance score. 

In still yet another enobodiment, the present invention includes a method, system, and computer 
5 readable medium storing computer executable instructions for evaluating an analyst's performance, and 
includes utilizing information pertaining to at least one upward or downward revision including an 
upward oc downward change in opinion of the analyst with regard to at least cme investment This 
embodiment also includes calculating a performance score indicative of the analyst's perfonnance 
relative to other analysts, said performance score determined at least in part by considering a measure 

10 of variability of the analyst's performance, an average historical perfonnance of the at least one 

investnaent following the at least one upward or downward revision, a standard deviation of the at least 
one upward or downward revision, a number of upward or downward revisions made by the analyst, 
and a likelihood that the at least one upward or downward revision will actually produce an expected 
result In addition, this embodiment includes comparing the analyst's perfonnance scoie against 

IS performance scares of other analysts to produce at least one of an upward, downward or combined 
revision ranking. 

In yet another embodiment, a method of evaluating an analyst's performance by utilizing 
information pertaining to at least one revision made by the analyst involving at least one investment is 

20 disclosed. In this embodiment, the invention includes determining a first average return value (ave. 
retumi), second average return value (ave. return^), and third average return value (ave. retuma) for the 
at least one revision. Each of the first, second, and third average return values corresponds respectively 
to an average rate of return on the at least one revision involving the investment for a first period of 
time (timeO, a second period of tune (time^), and a third period of time (times). From there, a first t-stat 

25 value (t-stati) for ave. return^ a second t-stat value (t-stat^) for ave. retum2, and a third t-stat value (t- 
stata) for ave. returns, are calculated, in absolute tenns. In particular, the first, second, and third t-stat 
values are calculated by utilizing ave. retumi. ave. rBtum2, ave. returns, a first standard deviation (SDi), 
a second standard deviation (SD2), and a third standard deviaticMi (SD3), a total number of revisions 
(revisionsi) occurring more than or eSqual to time] before said evaluating, a total munber of revisions 

30 (revisionsi) occurring more than or equal to time2 before said evaluating, and a total number of 

revisions (revisionsa) occurring more than or equal to times before said evaluating. Subsequently, a 
first degree of fireedom (DFi) for ave. retumi. a second degree of freedom (DF2) for ave. retumi, and a 
third degree of fteedom (DF3) for ave. returns are calculated. The invention then contemplates 
detmnining a first probability (probi) for timei by utilizmg t-stati and DFi, a second probability (probi) 

35 for time2 by utilizing t-stata and DF2, and a third probability (probs) for times by utilizing t-stats and 
DFs. A raw conditional perfonnance score for the analyst is generated by utilizing probj, probi, probs. 
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A conditional performance score (cond. performance score) may be generated by adding a modifier 
(adjustment), which may be determined by utilizing revisions u revisionsa, and revisionss, to the raw 
conditional performance score. From there, the invention contemplates generating a small return 
penalty by subtracting a penalty from an average of return factors corresponding to timei, time2, and 
5 time3^ wherein for upward revisions the penalty is 0 if ave. returns is greater than 15%, else the penalty 
is equal to 15% minus ave. returns, wherein for downward revisions the penalty is 0 if ave. returns is 
less than -8%, else said penalty is equal to -8% minus ave. returns, wherein the first period of time 
return factor is ten if ave. retumi is greater than ten, else the first period of time return factor is equal to 
a cube root of a product of 100 and ave. retumi, wherein the second period of time return factor is ten if 

10 ave. retum2 is greater than ten, else the second period of time return factor is equal to a cube root of a 
product of 100 and ave. retun^. and wherein the third period of time return factor is ten if ave. returns is 
greater than ten, else die third pmod of time return factOT is equal to a cube root of a product of 100 and 
ave. returns. Using the above factors, a final performance score (performance score) indicative of the 
analyst's performance may be generated by adding the small return penalty to the conditional 

15 performance score. 

There has thus been outlined, rather broadly, the more important features of the invention in 
order that the detailed description thereof that follows may be better understood, and in order that the 
present contribution to the art may be better appreciated. There are, of course, additional features of the 
20 invention that will be described hereinafter and which will form the subject matter of the claims 
appended hereto. 

In this respect, before explaining at least one embodiment of the invention in detail, it is to be 
understood that the invention is not limited in its application to the details of construction and to the 
25 arrangenffints of the components set fctfth in the following description or illustrated in the drawings. 
The invention is capable of other embodiments and of being practiced and carded out in various ways. 
Also, it is to be understood that the phraseology and terminology employed herein are for the purpose 
of description and should not be regarded as limiting. 

30 As such, those skilled in the art will appreciate that the conception, upon which this disclosure 

is based, may readily be utilized as a basis for the designing of other structures, methods and systems 
for carrying out the several purposes of the present invention. It is important, therefore, that the clainos 
be regarded as including such equivalent constructions insofar as they do not depart from the spirit and 
scope of the present invention. 



35 
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Further, the purpose of the foregoing abstract is to enable the U.S. Patent and Trademaric Office 
and the public generally, and especially the scientists, engineers and practitioners in the art who are not 
fanoiliar with patent or legal terms or phraseology, to detennine quickly firom a cursory inspection the 
nature and essence of the technical disclosure of the application. The abstract is neither intended to 
5 define die invention of the application, which is measured by the claims, nor is it intended to be limiting 
as to the scope of the invention in any way. 

These together with other objects of the invention, along with the various features of novelty 
which characterize the invention, are pointed out with pardcuiarity in the claims annexed to and 
10 forming a part of this disclosure. For a better understanding of the invention, its operating advantages 
and the specific objects attained by its uses, reference should be had to the accompanying drawings and 
descriptive matter in which there is illustrated preferred embodiments of the invention. * 

Other objects of the present invention will be evident to those of ordinary skill, particularly 
15 upon consideration of the following detailed description of the preferred embodiments. 

NOTATIONS AND NOMENCLATURE 

The detailed descriptions which follow may be presented in terms of program procedures 
20 executed on computing or processing systems such as, for example, a stand-alone gaming machine, a 
computer or network of computers. These procedural descriptions and representations are the means 
used by those skilled in the art to most effectively convey the substance of their work to others skilled 
in the art 

25 A procedure is here, and generally, conceived to be a self-consistent sequence of steps leading 

to a desired result These steps are those requiring physical manipulations of physical quantities. 
Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable 
of being stored, transferred, combined, compared and otherwise manipulated. It proves ccxivenient at 
times, principally fcH* reasons of conimon usage, to refer to these signals as bits, values, elements, 

30 symbols, characters, terms, numbers, or the like. It should be noted, however, that all of these and 
similar terms are to be associated with the appropriate physical quantities and are merely convenient 
labels applied to these quantities. 



35 



Further, the manipulations performed are often referred to in terms, such as adding or 
comparing, which are conmionly associated with mental operations performed by a human operator. 
No such capability of a human operator is necessary, ox desirable in most cases, in any of the operations 
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described herein which fonn part of the present invention; the operations are machine operations. 
Useful machines for perfonning the operation of the present invention include general purpose digital 
computers or similar devices. 



RRTFF DKSCRIPTION OF TEDE DRAWINGS 



FIG. 1 LUustrates a prior art stock and cash portfolio development system; 

FIG. 2 illustrates a prior art securities valuation system; 

FIG. 3 illustrates a prior art investment fund management system; 

FIG. 4 illustrates a prior art data processing system for selecting securities and constructing 
investment portfolios based on a set of artificial neural netwcxrks; 

FIGS. Sa-Sb depict one example of a user interface utilizable for displaying an analyst 
evaluation according to the principles of the present invention; 

FIGS. 5c-Sd depict one exanqple of a user interface displaying exanqiles of revisicns utilizable 
in conjunction with the principles of the present invention; 

FIG. 6 is a flow diagram illustzating a process for evaluating analysts acccnding to the 
principles of the present invention; 

FIG. 7 is a flow diagram illustrating a high level process for generating an upward revision 
performance measure utilizable in the process of FIG. 6; 

FIG. 8 is a flow diagram illustrating a high level process for genmting a downward revision 
performance noeasure utilizable in the process of FIG. 6; 

FIG. 9 is a flow diagram illustrating a process for generating an upward revision small return 
adjustment utiUzable in the processes of FIG. 7; 

FIG. 10 b a flow diagram illustrating a process for generating a downward revision small 
return adjustment utilizable in the processes of FIG. 8; 

FIG. 11 is a flow diagram illustrating a process for ranking analysts according to an upward 
revision perfcMmance measure utilizable in the process of FIG. 7; 

FIG. 12 is a flow diagram illustrating a process for ranking analysts according to a downward 
revision performance measure utilizable in the process of FIG. 8; 

FIG. 13 illustrates one example of a central processing unit for implementing a ccnnputer 
process in accordance with a computer iiiq)leDQented embodiment of the present invention; 

FIG. 14 illustrates one exaiiq)le of a block diagram of internal hardware of the central 
processmg unit of FIG. 13; 

FIG. IS illustrates another exan[q)le of a block diagram of internal hardware of the central 
processing unit of FIG. 13; 
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FIG. 16 illustrates one example of a memory medium which may be used for stoTing a 
computer implemented process of the present invention; 

FIG. 17 illustrates an example of a combined Internet, POTS, and ADSL architecture which 
may be used with the present invention; and 
5 FIG. 18 shows a set of tabulated results of an evaluation performed according to the principles 

of the present invention. 

BEST MODE FOR CARRYING OUT THE INVENTION 

10 Reference now will be made in detail to the presently prtfened embodiments of the invradoa 

Such embodinrats are provided by way of explanad(xi of the invention, which is not intended to be limited 
thereto. In fact, those of ordinary skill in the art may appreciate upon reading the present specificaticHi and 
viewing the (»esent drawings that various modifications and variations can be made. 

1 5 Fc3r example, features illustrated or described as part of one embodiment can be used on other 

embodiments to yield a still further embodimenL Additionally, certain features may be interchanged with 
similar devices or features not menticnaed yet which perfoan the same or similar functions. It is therefore 
intended that such modifications and variations are included within the totality of the present invention. 

20 In accordance with the principles of the present invention, an analyst's performance is 

evaluated by utilizing information pertaining to at least cxie revision made by the analyst involving at 
least one investment This evaluation includes determining a conditional performance score indicative 
of the analyst's performance relative to other analysts. The performance score is determined at least in 
part by considering an average historical performance of the investment following the revision. In 

25 addition, the perfonnance score is also determined using a historical consistency of the analyst's 

performances with respect to revisions involving the investment, and a number of revisions made by the 
analyst Then, the perfcnrmance score may be adjusted according to one or more adjustments to 
generate a final performance score. 

30 These performance scores may be determined based on upward or downward revisions or a 

combination of both. In particular, a score based on upward revisions, or an upward revision 
performance score, corresponds to the predictive nature of an analyst with respect to predicting or 
influencing increases in earning. In contrast, a score based on downward revisions, or a downward 
revision perfcxmance score, corresponds to the predictive nature of an analyst with respect to predicting 

35 or influencing decreases in earnings. Similarly, a combined score, or for example an average of the two 
previous scores, corresponds to the overall predictive nature of the analyst 
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Refecring first to FIGs. 5a and 5b, one example of a user interface 500 utilizable for displaying, 
for example, analyst performance scores is illustrated In FIG. 5, the user interface displays a wide 
variety of information including performance scores for any number of companies 5 10. To calculate 
the conopany perfoxinance scores in this example, the scores for each of the analysts covering a 
particular company are averaged to produce an overall company score. In addition to listing and 
determining an average of the perfOTnoance scores of all the analysts covering a company, any 
combination of performance scores may optionally be determined and listed For instance, an average 
of only the upward or downward revision performance scores for the analysts covering a particular 
company may be determined and listed. Alternatively, a user may just as easily elect to view a list of 
only the upward or downward revisions for a single company. Similarly, an index of bdividual analyst 
performance scores, independent of the companies, based either on upward or downward revisions or 
both may also be determined and listed 

Advantageously, these performance scores are a measure of the predictive nature of each 
analyst's revisions. In one exannple, a revision is a change in the c^inion of an analyst on a 
performance of a company or other entity, for example, overall conc^any revenues, profits, market 
share, and/or how much a company is likely to earn per share in a future quarter, fiscal year, or other 
time period. For instance, as shown in FIGs. 5c-5d, other examples of revisions utilizable with 
embodiments of the present invention include detailed analyst recommendations on whether to sell, 
hold or buy a security 530, earnings surprises 532, estimates submitted 534, price/earnings 536, 
monaentum, price/earning ratio, P/E and growth rates, and/or PEG ratio, and/or other similar 
infcHmation/estimates. Some causes for an upward revision include: the release of a good quarterly or 
annual earnings report, a positive analysts* noeeting or conference call with company executives, or 
improvements at a similar company in the same industry. Likewise, causes for a downward revision 
include: the release of a grim quarterly or aimual earnings report, a negative analysts* meeting or 
conference call with company executives, or worsening results at a similar company in the same 
industry. 

In the [»esent invention, the higher the score of the analyst (with one-hundred, for example, 
being the highest), the more predictive on a historical basis. Furthermore, the performance seem 
increases with the number of beneficial revisions of the analyst For mstance, the perfcKmance score 
increases when a rate of return of an investnoent increases and the revision is an upward revision, or 
when a rate of return of the investment decreases and the revision is a downward revision. Likewise, 
the perfonnance score decreases when a rate of return of the mvestment decreases and the revision is an 
upward revision, or when a rate of return of the investment increases and the revision is a downward 
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revision. Thus, a high upward revision perfonnance score corresponds to historically high positive 
returns (e.g., the analyst issued an upward revisicm, and the stock went up in value). Similariy, a high 
downward revision performance score conresponds to high negative returns (e.g., the analyst issued a 
downward revision, and the stock went down in value). In addition, it should be noted that although in 
5 this particular embodiment corporate secxirities are mentioned as being one exaiEple of the analyst's 
revisions, the instant invention is also applicable to and should be construed as encompassing any other 
form of similar investment. Some examples include bonds, currencies, conomodities, paper, precious 
nietals, minerals, natural resources, mutual funds, and the like or any other analogous investments 
and/or investoaent entities. 

10 

As will be discussed in greater detail below, several factors are incorporated into these 
performance scores. For instance, an analyst's average return history (e.g., a return amount typically in 
a percentage calculated over a predetermined period of time after the analyst' s revision) for both 
upward and downward forecast revisions may initially be considered. ladeed in alternative 

15 embodiments, this factor is accorded the greatest amount of weight Next, the analyst*s historical 
consistency at issuing upward and downward revisions in a company's stock, as measured by, for 
example, standard deviation and the like are also considered In a related marmer, a measure of 
variability of the analyst's performance may also be considered. Similarly, a likelihood that the 
revision will actually produce an expected result, as embodied by statistical characteristics such as 

20 probability and the like are also factored into the calculation. Finally, the number of revisions that 
make up an analyst's track record is factored into the performance score for the simple reason that the 
more times an analyst has demonstrated good judgment the better. Other factors are optionally used as 
well. 

25 As to the analyst's average return history, any number of return periods may be considered. 

For example, in one embodiment, two periods are utilized. In other embodiments, three or noore 
periods may be used. Furthermore, any length of tinae is possible. Thus, in some cases, two periods 
(e.g., a relatively shcHt period of time and a relatively long period of time) suffices. In other cases, one 
or more intermediate periods of tixnef may also be included into the calculation. As examples, three 

30 month and six month periods may be used. As another example, five, twenty, and sixty day periods 
may be used. Similarly, one week, one month, and two month periods arc also possible. 

In addition to the above, other factors may optionally be considered as weU. For example, the 
above factors may be utilized to generate a conditional or raw p^ormance score, which in turn may be 
35 modified or adjusted to produce a final performance score. For instance, a nxxiifier accentuating the 
number of revisions made by the analyst may be factored into the conditional performance score. In 
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othCT cases, a rctuni amount adjustment may be made to reward or penalize an analyst for small or large 
return amounts. For example, the return anoount adjustment may include a small return penalty 
penalizing or reducing the analyst's performance score for returns less than a predetermined amount 
More specifically, a penalty may be imposed for returns of less than a predetermined anaoimt as 
5 calculated over a predetermined period of time. Hence, with upward revisions, a penalty might be 
Liiq)osed for returns of less than approximately 15% as calculated over approximately a two month 
period of tinae. In contrast, with downward revisions, a penalty might be imposed for returns greater 
than -S% as calculated over a two month period of time. Likewise, high returns may also result in 
increases to the analyst*s performance score. Further, adjustments need not be limited to return 
10 amounts. As an example, adjustments may be made to a performance score for an extremely large 

number of accurate trades. Indeed, the present invention contemplates modifications or adjustments for 
nearly any factor utilizable for predicting accuracy or reliability. 

Returning to the interface displayed in FIG. 5, in addition to listing the average scores for all of 
15 the analysts associated with a particular company, the interface may also be used to list each of the 
highest performing analysts for categories pertaining to downward and upward performance scores. 
Furthermore, interface 500 may also be linked to or used to display infofination concerning each 
analyst and/or the corresponding security. Likewise, the interface sponsor may optionally include 
commentary concerning, for exan:q)le, any securities or analysts, advertisements utilizable as a source 
20 of revenue, instructions or help information for beginning users, and information concerning or tracking 
die activities of exceptionally higji performing analysts, or other similar features. 

While the interface depicted in FIG. 5 is used to view the average of the performance scores 
associated with a particular list of coi]:q)anies, other listings are also possible. For example, a user may 

25 select a particular analyst for review. Li this case, each of the upward, downward, aiul combined 

performance scores for that analyst may be viewed. Furthermore, if the selected analyst is responsible 
for monitoring more than one conaf>any, multiple values for upward, downward, and combined 
performance scores (e.g., one score for each company) may be listed. Similarly, a user may select any 
combination of analysts or sets/groupings of data including, for example, all analysts dealing with or 

30 investing in a particular industry or market, or other analogous listings of information. 

A broad overview of the process of one embodiment of the present invention for calculating 
performance scores is now illustrated with reference to FIG. 6. First, a list of analysts is retrieved from 
one or any number of databases 610. As will be discussed in greater detail below with reference to 
35 FIG. 17, the database may include a list of any or all of a variety of combinations of analysts. Fc^ 
exanople, all of the analysts tracking conq)anies in a single industry may be included Similarly, the 
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database may include all insiders filin g insider documents with the SEC. In other embodiments, other 
similar or analogous techniques may be utilized to generate a list of analysts. For example, any one of a 
variety of databases tracking industry leaders may just as easily be used. 

After retrieving the list of analysts, a smaller list of analysts to be evaluated is optionally 
generated according to predetermined criteria. In this regard, a check is performed identifying analyst 
that meet predetermined criteria 615. In this example, any number of criteria may be used to generate 
this list For example, each analyst may be required to be an actual living person, thereby excluding 
brokerage fiams. Thus, m this scenario, any entities that are not people, such as companies, 
partnerships, or trusts, are removed from the evaluation list As another example, the market 
capitalization of the company being monitored may be required to be greater than a predetennined 
amount, such as, for instance, $50 million. Third, each analyst may be required to have issued a 
miniTniim number of revisions that have been in existence for a certain period of time. For instance, 
some embodiments may require analysts to have issued at least two revisions, with each revision being 
at least sixty days old. Other embodiments may require more or fewer revisions, which in turn may be 
more recent or older than sixty days. These requirements are disclosed for exenaplary purposes only 
and it is to be understood that other types of analyst requirements are possible within the scope of the 
invention. For instance, non-people analysts such as partnerships, corp<»ations, and other joint ventures 
or entities are also possible where groups of analysts participate in recommendations and/or decision 
making. Thus, a group of, for exan:q>le, Morgan Stanley analysts may be conq)ared against Goldman- 
Sachs analysts, individually or overall. Similarly, entities not having market capitalization or trading 
limits are also possible. Furthennore, the concepts of the present invention are applicable to entities 
investing in investments other than securities, such as bonds, mutual funds, or any other similar 
investments, trading securities, investments and the like. 

If the analyst does not meet the predetermined criteria (615), a message is returned, aiKl the 
analyst is not evaluated 620. On the other hand, if the analyst meets the predetermined crit^ia, 
processing continues with the calculation of a performance score for the analyst 625. As will be 
discussed below, this performance score may reflect upward revisions, downward revisions, or a 
combination, and/cff average distribution of the two. From there, processing continues with a 
determination of whether more analysts exist to be ranked 630. 

If more analysts are to be ranked, processing returns to 610. If the list of analysts to be ranked 
is complete, each of the analysts meeting the predetermined criteria are ranked 635. Subsequently, 
these rankings may optionally be disseminated 640 via, for example, a user interface similar to that 
depicted in FIG. 5 in a local cx' global setting, e.g., intranets, the Intemet, and the like. Alternatively, 
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the rankings way be disseminated in any other similar or analogous medium, such as in a magazine or 
newspaper, a trade journal, television, and radio or the Uke. 

Although in this embodiment a list of analysts to be evaluated is determined according to 
5 predetermined criteria, it is to be understood that other alternatives are also possible. For instance, the 
present invention is to be construed as including scenarios where a user enters or selects a 
predetermined listing of analysts for evaluation. In such a case, only the particular entities entered by 
the user would be evaluated. It is also possible that a mixed grouping of investment entities and types 
may be evaluated. For instance, it is possible within the scope of the present invention to evaluate 
10 corporate insiders, nonperson joint ventures, brokerage firms, and/or corporations, and any other types 
of investment entities or combinations thereof. 

The calculation of an upward revision performance score is now described with reference to 
FIG. 7. In this example, one or more rates of return on an investment corresponding to an analyst's 

15 upward revision, taken a predetermined period of time after issuance of the upward revision, are 

utilized in the generation of the performance score. Thus, one or mote return values for one or more 
return intervals are initially calculated or gathered for each analyst 705 by any suitable means. In this 
particular embodiment, three return values, one for a short period of time, one for an intermediate 
period of time, and one for a long period of time, are taken. As to the actual amount of time associated 

20 with each period, any lengths may be used. For example, the short period of time may be five days or a 
week, the intermediate penod may be twenty days or a month, and the long period may be sixty days or 
two months. Furthermore, three return values are utilized in this embodiment for exemplary purposes 
only. In actual practice any number of return values, at any number of time intervals, are allowed so 
long as they are suitable proxies for, e.g., short, long, and optionally intermediate periods of time. As 

25 an example, two return values, one at 3 nKmths (e.g., 13 weeks) and a second at 6 months (e.g., 26 
weeks), may just as easily be utilized. 

In accordance with the principles of the present invention and as discussed briefly above, these 
scores or measures reflect an analyst's performance and are generated by considering a number of 

30 performance factors. One embodiment of the present invention uses a measure of the analyst's 

historical performance as determined by rates of return over certain periods of time after a revision. 
Another example may include considering the analyst's success at selecting entry and exit pcdnts in the 
investment Additionally, as will be discussed below, the analyst's performance score may be modified 
or weighted based on a number of revisions made by the analyst, the magnitude of a return, or other 

35 similar factors. Of course, the above factors are listed for exenoplary purposes only and it is to be 
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understood that other similar and analogoxis factors may also be considered so long as they provide an 
accurate and/or estimated ineasurement of the analyst's success or lack tbereof. 

After calculating the return values, a statistical measure of confidence in the analyst's 
performance may be calculated for each return value 710. As one exanq)le, a t-statistic (x t-stat, taken 
in absolute terms, is utilized as this measure of confidence. The t-stat is a measure on a random sample 
(or pair of samples) in which a mean (or pair of means) appears in the numerator, and an estimate of the 
numerator's standard deviation appears in the denominator. The latter estimate is based on the 
calculated san^le variance estimates of the samples. Alternatively, other standard variance measiffes or 
other suitable indices of variability may optionally be used in place of standard deviation, and/cw t-stat. 
If these calculations yield a value of (t) that is sufficiently different firom zero, the test is considered to 
be statistically significant 

In this embodiment, the t-stats for each retum period may be determined by: 

upward revision retum 



upward revision t stat = 



upward _ revision _ SD 



^ total _ number ^of _ upward _ revisions 
In this example since three intervals are being utilized, three t-stat values are calculated, one for each 



interval. 



To prevent division by zero, if the standard deviations for any of the time intervals arc zero, 
then the conditional upward revision performance score for that interval is set to zero. In addition, 
although the t-stat is used m this exanq)le, other measures of confidence may just as easily be 
incorporated in the present invention. As another example, the z-statistic or other similar measurements 
of variability, dispersion, or consistency may be used. Furthermore, reganlless of what method is used 
to produce the measure of confidence, the measure of confidence may also be manipulated or weighted 
to consider other factors, so long as it accurately reflects or estimates, for instance, the analyst's retum 
history and historical consistency at issuing forecast revisions in the investment. 

After calculating the measures of confidence (e.g., the t-stats for each time interval or retum 
period), one or more degrees of fi^edom are determined by subtracting, for example, one from the 
number of revisions for each return value 715. Hence, with three return values, three degrees of 
freedom are generated. The degrees of freedom are used to describe the number of values in a final 
calculation of a statistic that arc fiw to vary, and measure, for exaii5)le, repetidon, repeatability, and 
certainty. Hence, each degree of freedom is determined by: 
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upward revision degree of fteedom = total number of upward re\dsions - 1 

Next, a probability indicating, for exanq)le, the likelihood that an analyst's and/or other 
investing entity's actions will actually produce the expected result, is determined for each return 
interval 720. Thus, with three return int^ais, three probabilities are generated. In this example, the 
degrees of fixedom, along with the absolute t-stats, are used in determining a probability for each return 
period 720. Specifically, subject to the following optional conditions, the probabilities are looked-up in 
a common statistics lookup table using each return intervales degree of freedom and t-stat If any of the 
degrees of freedom are greater than 100, then that degree of freedom is set to 100 before looking up the 
probability. If any of the absolute t-stats are greater than 20, then that absolute t-stat is set to 20 before 
looking up the probability. Finally, if any of the absolute t-stats are less than 0.01, then the 
conesponding probability is siiiq)ly set to 0. In addition, the probabilities may be calculated utilizing 
other methods. For instance, the t-stat and the degree of freedom noay undergo further manipulation 
before looking up the probability. Alternatively, other analogous methods may be utilized in place of 
the look-Hip table to determine the probabilities. 

Subsequent to determining the probabilities for each return period, a conditional upward 
revision performance score, or raw conditional upward revision performance score, is generated 725. 
Although odier methods are possible, the conditional performance score in this example is determined 
by: 



cond _up^ per „ score = 



[l-(.5 + or-up^ probl)] + [l - (.5 + or - ifp . prob2)] + [l - (.5 + or - up , prob3)] 



In this formula, up_probl, up_prob2, up__prob3 conespond respectively to the probability for a first, 
second, and third return interval. Referring again to die above formula, whether the probabilities are 
added or subtracted is determined by the following conditions. If any of the three probabilities are 
greats than zero, then that probability is subtracted fromO.S. Otherwise, O.S is added to the 
probability. 

The conditional upward revision performance score represents an initial analyst performance 
measure with respect to upward revisions and may undergo one or noore of any number of 
manipulations to result in various other refined performance measures and ultimately a final upward 
revision perfcnmance score. As an example, the ccHiditional performance score may be weighted to 
place additional emphasis on the maturity of the corpcxation. Alternatively, investments in, for 
example, conservative sectors may also receive a noodification. To more cleariy illustrate, exen[q)lary 
modifiers accounting for the size of a return and a number of revisions issued are discussed in greater 
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detail below. These nxxiifications are intrcxluced for exemplary reasons only and it is to be understood 
that other manipulations are also possible within the scope of the present invention. 



Initially, an optional upward revision adjustment may be generated to reQect and emphasize the 
number of revisions issued by the analyst Specifically, a nKxlLfication may be made to the conditional 
upward revision performance score accentuating the number of upward revisions issued by the analyst 
730. 



In this example, the adjustment is determined by: 

I 



up. rev. adj.= (-20)* 



num_up_revl-^num_up_rev2 + num_up_rev3 



In the above formula, nunL-up_revl, num„up_rev2, and num_,up_rev3 correspond respectively to the 
number of upward revisions for a first interval, the number of upward revisions for a second interval, 
and the number of upward revisions for a third interval. As will be noted fixjm the above formula, three 
separate upward revision numbers or values for three dme intervals are considered in this embodiment 



As another exanq>le, an upward revision return amount adjustment may be generated to 
emphasize a large or small return amount after a revision forecast made by the analyst This adjustment 
may be utilized to penalize an analyst for a small return, or reward an analyst for a large return. 
Referring to FIG. 9, one example utilizable for determining a return amount adjustment f<x an upward 

20 revision is illustrated. In this embodiment, the return amount adjustment includes a small return penalty 
for reducing the conditional performance score for returns on the investment that are less than a 
predetermined amount Although any predetermined amount may be utilized, with upward revisions in 
this example, it is set to approximately 15%. The small return penalty is generated using an average of 
a number of return factors mi a penalty amount More particularly, the small return penalty is 

25 detOTnined in one embodiment by taldng a difference between an average of a short period of time 

return factor, an intermediate period of time return factor, and a long period of time return factor, and a 
penalty. Thus, each of a short period of time return factor, an intermediate period of time return factor, 
and a long period of time return factor are initially calculated 905, 910, and 915. 



30 



In this embodiment, subject to certain conditions discussed below, each of the period of time 
retum factors (e.g., short, intermediate and long return factors) is equal to a cube root of a return 
percentage over a corresponding period of time, unless the return is greater than approximately 10 or 
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1000%. If the return is greater than approximately 10 or 1000%, the factor is simply set to 10 or 
1000%. Thus to restate the above, if the average return for the period of time is greater than 10, then 
the return factor is simply set to 10. However, if the average return for a period of time is not greater 
than 10, the return factor is equal to: 

5 

return factor = (100 * average upward revision return)^ 

After calculating each of the return values, the optional penalty is generated 920. In this 
embodiment, the penalty is a difference between a predetermined amount and a return for a 

10 predetermined period of time. The predetermined period of time is optionally equal to a time 

coirespooding to the long period of time return factor. If the average return value corresponding to the 
investment fot the predetermined long period of time is greater than a predetermined amount, the 
penalty is sinq>Iy set to 0. As an example, the period of time and the predetermined amount may be 
approximately sixty days and approximately 0.15 or 15%, respectively. Thus in this example, a penalty 

15 is calculated only if the return over a sixty day period of time is less than 15%. However, if the average 
return value corresponding to the investment for the long period of time is not greater than 0. 15 ox 15%, 
the penalty is equal to a product of 100 and a difference between 0.15 (or 15%) and the average return 
value corresponding to the investment for a sixty day period of time. In other words, if the average 
return value corresponding to the investment for the long period of time (e.g., sixty days) is not greater 

20 than 0.15 or 15%: 



penalty = (100 * (0.15 - average upward revision return for the long period of time)) 

This penalty is then subtracted from the average of the return factors 925 (discussed above) and 
25 used to produce an optional adjustment- 
adjustment = (short return factor + intermediate return factor + long return factor)/3 - penalty 

Thus, in this example, a small return penalty is imposed for small returns (e.g., approximately 
30 15%) over a long period of time (e.g., approximately sixty days) after a revision. Once the return factors 
and penalty have been determined and used to produce an adjustment, the adjustment(s) is added to the 
conditional performance score to produce a final upward revision performance score 735. 



35 



In the above example, adjustments reflecting the number of revisions issued by the analyst and 
reflecting return an[iounts emphasizing a small retum amount are illustrated. However, as discussed 
above, any other noodificadons are also possible. For instance, nuxiifications may be made placing a 
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much greater emphasis on short term investments to generate an adjustment aimed at producing scores 
for use m formulating short term investment strategies. Or, in contrast, greater weight may be placed 
on returns over longer periods of time to reflect, for example, a more ccxiservative invesdnent strategy. 

5 The generation of performance scores based on downward revisions is sunilar to the generation 

of upward revision performance scenes and is depicted in FIG. 8. However, in contrast to upward 
revision performance scores, downward revision performance scores are indicative of an analyst's 
influence or accuracy in predicting ttie performance of, for example, a security after issuance of a 
downward revision. A conditional downward revision performance score may be manipulated in any 
10 number of ways to produce refined or final downward revision performance scores. 

As with the upward revision sccxes, processing commences with the calculation or 
determination of one or more return values, oc rates of return, for each analyst, taken a predetermined 
period of time after issuance of the downward revision 805. Again, any number of return values may 
15 be taken. In this particular embodiment, three return values, one for a short period of time, one for an 
intermediate period of time, and one for a long period of time, are taken. Furthermore, any lengths may 
be associated with these periods of time (e.g., a sh<Kt period of time of five days, an intermediate period 
of twenty days, and a long period sixty days). While, three return values are utilized in this 
embodiment, any number of return values, at any number of time intervals, are allowed. 

20 

Subsequently, a statistical measure of confidence in the analyst's performance is calculated 
810. In this case, a t-statistic (t-stat), in absolute terms, is again utilized for exemplary reasons, and is 
likewise calculated for each return value. Again, since three intervals are utilized in this example, three 
t-stat values are calculated, one for each interval. 

25 

For downward revisions, the t-stats for each return period are determined by: 



downward revision t stat = 



downward revision return 



downward _ revision _ SD 
^ total _ number ^of _ downward _ revisions 



30 



Like with the upward revision performance score, to prevent division by zero, if the standard deviations 
for any of the three time intervals are zero, then the downward revision performance score for that 
interval is set to zero. 
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Again, as with the above example, it is to be understood that other measures of ccxifidence may 
be substituted for the t-staL Also, any number of modifications are possible. As indicated above, as 
long as an accurate reflection of the analyst's retuni history and/or historical consistency at issuing 
downward revisions is considered, any measure of confidence, for example a z-stat or other similar 
5 measurement of variability, dispersion, or consistency may be used. 

After calculating the measures of confidence (e.g., the t-stats for each time interval or return 
period), one or more downward revision degrees of freedom are determined by subtracting, for 
example, one from the number of revisions for each return value 815. Hence, with three return values, 
10 three degrees of freedom are generated. Each degree of freedom is determined by: 

downward revision degree of freedom = total number of downward revisions - 1 

Subsequently, a probability indicating, for example, the likelihood that an analyst's and/or other 
15 investing entity's actions will actually produce the expected result, is determined for each return 

interval 820. In this example^ the degree of freedom values, along with the absolute t-stat values, arc 
used in determining a probability for each return period. Specifically, subject to the following optional 
conditions, the probabilities are looked-up in a common statistics lookup table using each return 
interval's degree of freedom and t-stat If any of the degrees of freedom are greater than 100, then that 
20 degree of freedom is set to 100 before looking up the probability. If any of the absolute t-stats are 

greats than 20, then that absolute t-stat is set to 20 before looking up the probability. Finally, if any of 
the t-stat are less dian .01, then the ccHresponding probability is simply set to 0. Alternatively, the 
probabilities may be calculated based on other the methods. Fac example, the measures of confidence 
and degrees of freedom may be modified or manipulated before looking up the probability in the 
25 lookup table. Also, any othw similar methods may be used in place of the look-up table to generate a 
probability. 

Subsequent to determining the probabilities for each retum period, a conditional downward 
revision performance score, or raw conditional downward perforcoance score, is generated 825. 
30 Although other methods are possible, the conditional performance sccffe in this example is determined 
by: 

, ^ {[l-i5-hor-dw^ probl)] + [l - (.5 + or - t/w _ probl)] + [l - (.5 + or - ifur „ prob3)] 
cond _ OH' _ per _ score - \ ^ 



*1C 



35 



In this fcOTMila, dw_probl, dw_prob2, dw_prob3 correspond respectively to the probability for a first, 
second, and third retum interval. Whether the probabilities are added or subtracted is determined by the 
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following conditions. In contrast to upward revisions, if any of the three probabilities are less than 
zero, then that probability is subtracted firom 0.5. Otherwise, 0.5 is added to the probability. 

As with the above, the conditional downward revision performance score represents an initial 
analyst performance measure and may undergo one or more of any number of manipulations to result in 
various other refined performance measures and ultimately a final downward revision performance 
score. As examples, modifiers accounting for the size of return and the number of revisions issued are 
discussed in greater detail below. These modifications are introduced for exemplary reasons only and it 
is to be understood that other manipulations are also possible within the scope of the present invention. 

For example, a downward revision adjustment may be generated to reflect and emphasize the 
nunob^ of revisions issued by the analyst Specifically, a modification may be made to the conditional 
downward revision perfOTmance score accentuating the number of downward revisions issued by the 
analyst 



In this example, the adjustment is determined by: 



dw. rev. adj. = (-20)* 



num_dw_r€vl + num_dw_rev2'¥ni£m_dw_rev3 



In the above formula, num_dw_revl, num_dw_rev2, and num_dw_rev3 correspond respectively to the 
number of downward revisions for a first interval, the number of downward revisions for a second 
interval, and the number of downward revisions for a third interval. As will be noted fi'om the above 
formula, three separate downward revision numbers or values for three time intervals are considered in 
this embodiment 



As another exan^le, a downward revision return amount adjustment may be generated to 
emphasize a large or small return amount after a revision forecast made by the analyst This adjustment 
may be utilized to penalize an analyst for a small return, or reward an analyst for a large return- 
Referring to FIG. 10, one example utilizable for determining a return amount adjustment for a 
downward revision is illustrated. Li this embodiment, the return amoimt adjustment includes a small 
return penalty for reducing the conditional perfcmnance score for returns on the investnoent that are less 
than a predetermined amount Although any predetermined amount way be utilized, with downward 
revisions, in this example, it is set to approximately negative 8%, The small return penalty is generated 
using an average of a number of return factors and a penalty amount. More particularly, the small 
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return penalty is determined by taking a difference between an average of a short period of tinoe return 
factor, an intennediate period of time return factor, and a long period of time return factor, and a 
penalty. Thus, each of a short period of time return factor, an intermediate period of time return factor, 
and a long period of time return factor are initially calculated 1005, 1010, and 1015. 

5 

In this embodiment, subject to certain conditions discussed below, each of the period of time 
return factors (e.g., short, intermediate and long return factors) is equal to a cube root of a return 
percentage over a corresponding period of time, unless the return is greater than approximately 10 or 
1000%. If the return is greater than approximately 10 or 1000%, the factor is simply set to 10 or 
10 1000%. Thus to restate the above, if the average return foe the period of time is greater than 10, then 
the return factOT is simply set to 10. However, if the average return for a period of tinae is not greater 
than 10, the return factor is equal to: 

return factor = (100 * average downward revision return)^ 

15 

After calculating each of the return values, the penalty may be generated 1020. In this 
embodiment, the penalty is a difference between a predetermined amoxmt and a return for a 
predetermined period of time. In this exarr^le, the/ predetermined period of time may be (equal to a time 
corresponding to the long period of time return factor. If the average return value cOTesponding to the 

20 investment for the predetermined period of time is less than a predetermined amount, the penalty is 
simply set to 0. As an example, the period of tinoe and the predetermined amount may be 
approximately sixty days and approximately negative 0.08 or -8%, respectively. Thus in this example, 
a penalty is calculated only if the return over a sixty day period of time is greater than negative 8%. 
However, if the average return value corresponding to the investment for the long period of time is not 

25 less than negative 0.08 or -8%, the penalty is equal to a product of 100 and a difference between 

negative 0.08 (or -8%) and the average return value corresponding to the investment for a sixty day 
period of time. In other words, if the average retum value corresponding to the investment for the long 
period of time (e.g., sixty days) is not less than negative 0.08 or -8%: 

30 penalty = (1(X) * (-0.08 - average downward revision retum for the long period of time)) 

This penalty is then subtracted ftom the average of the return factors 1025 (discussed above) 
and used to produce the adjustment: 



35 



adjustment = (short retum factor + intermediate retum factor + long return factor)/3 - penalty 
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Thus, in this example, a small return penalty is imposed for small returns (e.g.. approximately - 
8%) over a long period of time (e.g., approximately sixty days) after a revision. Once the return factors 
and penalty have been determined and used to produce an adjustment, the adjustment(s) are added to 
the conditional perfOTmance score to produce a final downward revision performance score 835. 

5 

In accordance with the principles of the present invention, one example of a process for ranking 
analysts according to upward revision performance score is depicted with reference to FIG. 1 1. In 
particular, this process commences by sorting the analysts into descending (or ascending) upward 
revision performance score ordCT 1 105. Subsequendy, as wiU be discussed in greater detail below, the 
10 analysts are separated into a number of groups 1 110. Each group, dien, is assigned a ranking 1115, 
with, for example, the group with the highest scores receiving a ranking of 100 (signifying the best 
perfomiance) and the group with the lowest scores receiving a ranking of 0 (signifying the worst 
perfctmance). Thus, each analyst receives a rank equal to his or her group. 

15 Any method or procedure may be utilized to separate or divide the analysts into groups. In one 

embodiment, the analysts are simply divided to form a standard bell curve distribudon. In other 
embodiments, other distributions are j>ossible. As one example, the group of analysts is divided into 
seven groups based on upward revision performance score. 

20 The first of these seven groups includes the top 5% of the analysts as calculated according to 

upward revision performance score. The analysts falling into this first group are then divided into ten 
substantially equally sized subgroups. The highest of these subgroups is assigned a rank of 100. The 
next subgroup receives a rank of 99. This process continues imtil the last subgroup of the top 5% group 
is assigned a rank of 91. 

25 

The second group includes the next 12.5% of the analysts as calculated according to upward 
revision performance score (e.g., 5% - 12.5%). The analysts falling into the second group are then 
divided into fifteen substantially equally sized subgroups. The highest of these subgroups is assigned a 
rank of 90. The next subgroup receives a rank of 89. This process continues until the last subgroup of 
30 the second group is assigned a rank of 76. 



The third group includes the next 18% of the analysts as calculated according to upward 
revision perfOTmance score (e.g., 12.5% - 18.0%). The analysts falling into the third group are divided 
into fifteen substantially equally sized subgroups. The highest of these subgroups is then assigned a 
35 rank of 75. The next subgroup receives a rank of 74. This process continues until the last subgroup of 
the second group is assigned a rank of 61. 
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The fourth group includes the next 29% of the analysts as calculated according to upward 
revision perfOTmance score (e-g., 18% - 64.5%). The analysts falling into the fourth group axe divided 
into twenty substantially equally sized subgroups. The highest of these subgroups is then assigned a 
5 rank of 60. The next subgroup receives a rank of 59. This process continues until tbe last subgroup of 
the second group is assigned a rank of 41. 

The fifth group includes the next 18% of the analysts as calculated according to upward 
revision performance score (e.g., 64 J% - 82.5%). The analysts failing into the fifth group are then 
10 divided into fifteen substantially equally sized subgroups. The highest of these subgroiq)S is assigned a 
rank of 40. The next subgroup receives a rank of 39. This process continues until the last subgroup of 
the second group is assigned a rank of 26. 

The sixth group includes the next 12.5% of the analysts as calculated according to upward 
15 revision performance score (e.g., 82.5% - 95%). The analysts falling into the sixth group are divided 
into fifteen substantially equally sized subgroups. The highest of these subgroups is then assigned a 
rank of 25. The next subgroup receives a rank of 24. This process continues until the last subgroup of 
the second group is assigned a rank of 1 1. 

20 The seventh or last group includes the last 5% of the analysts as calculated according to upward 

revision performance score (e.g., 95% - 100%). Tbe analysts falling into the seventh group are then 
divided into ten substantially equally sized subgroups. The highest of these subgroups is then assigned 
a rank of 10. The next subgroup receives a rank of 9. This process continues until tlie last subgroup of 
the second group is assigned a rank of 1. 

25 

As depicted m FIG. 12, a similar procedure is utilized to rank the analysts with respect to 
downward revision p^onnance scoies. Thus this process is only briefly summarized here. Initially, 
the process commences by sorting the analysts into descending (or ascending) downward revision 
performance score order 1205. Subsequendy, the analysts are separated into a number of groups 1210. 
30 Then each group is assigned a ranking 1215, with, for example, the group with the highest scores 
receiving a ranking of 100 (signifying the best performance) and the group with the lowest scores 
receiving a ranking of 0 (signifying the worst performance). Like with the upward revision scores, each 
analyst receives a rank equal to his or her group. 

35 Similar to the upward revision procedure used to separate the analysts, any method may be 

utilized to divide these sorted analysts into groups. As one exan^le, the group of analysts is divided 
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into seven groups based on downward revision performance score. For instance, the first group may 
correspond to the top 5% and receive ranks ranging from 100-91. The second group may correspond to 
the next 12.5% of the analysts and receive ranks ranging from 90-76. The third group may correspond 
to the next 18% of the analysts and receive ranks ranging from 75-61. The fourth may correspond to 
5 the next 29% of the analysts and receive ranks ranging from 60- 41. The fifth may correspond to the 
next 18% of the analysts and receive ranks ranging from 40-26. The sixth group may correspond to 
the next 12.5% of the analysts and receive ranks ranging &x>m 25-1 1. The seventh or last group may 
correspond to the last 5% of the analysts and receive ranks ranging from 10-1. 

10 In addition, as noendoned above, any other similar and analogous methods may be used to 

separate and/or divide the analysts into groups. For instance, each group may be required to include an 
equal number of analysts. Similarly, different criteria may be used to determine the groups. 

One example depicting the tabulated results of processing in accordance with the above 
15 techniques with respect to a group of analysts covering a single company (IBM) is shown in FIG. 18. 
In this example, both upward and downward performance scores are hsted. Also, as discussed above, 
the scores may be averaged or otherwise combined to yield a combined performance score. Other 
alternatives are also possible, 

20 Viewed externally in FIG. 13, a coiiq)uter system designated by reference numeral 140 has a 

computer 142 having disk drives 144 aiKl 146. Disk drive indications 144 and 146 are merely symbolic 
of a number of disk drives which might be acconomodated by the computer systenL Typically, these 
would include a floppy disk drive 144, a hard disk drive (not shown externally) aixl a CD ROM 
indicated by slot 146. The number and type of drives vary, typically with different computer 

25 configurations. Disk drives 144 and 146 are in fact optional, and for space considerations, are easily 
omitted fix)m the computer system used in conjunction with the production process/apparatus described 
herein. 

The conqjuter system also has an optional display upon which information, such as the screen 
30 illustrated in FIG. 5 may be displayed. In some situations, a keyboard 150 and a mouse 152 are 
provided as input devices through which a user's actions may be inputted, thus allowing input to 
interface with the central processing unit 142. Then again, for enhanced portability, the keyboard 150 
is either a limited function keyboard or omitted in its entirety. In addition, mouse 152 optionally is a 
touch pad control device, or a track ball device, or even omitted in its entirety as well, and similarly 
35 may be used toinput a user's selections. In addition, the computer system also optionally includes at 
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least one infiraied transinitter and/or infirared received for cither transmitting and/or receiving infrared 
signals, as described below. 

FIG. 14 illustrates a block diagram of the internal hardware of the computer system 140 of FIG. 
5 13. A bus 156 serves as the main information highway interconnecting the other components of the 
computer system 140. CPU 158 is the central processing unit of the system, performing calculations 
and logic operations required to execute the processes of the instant invention as well as other 
programs. Read only memory (ROM) 160 and random access memory (RAM) 162 constitute the main 
memory of the conaputer. Disk controller 164 interfaces one or more disk drives to the system bus 156. 
10 These disk drives are, for example, floppy disk drives such as 170, or CD ROM or DVD (digital video 
disks) drive such as 166, or internal or external hard drives 168. As indicated previously, these various 
disk drives and disk controllers are optional devices. 

A display interface 172 interfaces display 148 and permits information from the bus 156 to be 

15 displayed on the display 148. Again as indicated, display 148 is also an optional accessory. For 
exanaple, display 148 could be substituted or omitted. Communications with external devices, for 
example, the other components of the system described herein, occur utilizing communication port 174. 
For example, optical fibers and/or electrical cables and/or conductors and/or optical communication 
(e.g., infrared, and the like) and/or wireless communication (e.g., radio frequency (RF), and the like) 

20 can be used as die transport medium between the external devices and communication port 174. 
Peripheral interface 154 interfaces the keyboard 150 and the noouse 152, permitting input data to be 
transmitted to the bus 156. In addition to the standard conqKments of the computer, the computer also 
optionally includes an inj&ared transmitter 178 and/or infrared receiver 176. Infrared transmitters arc 
optionally utilized when the computer system is used in conjunction with one or more of the processing 

25 components/stations that transmits/receives data via infrared signal transmission. Instead of uriliring an 
infrared transmitter or infrared receiver, the computer system may also optionally use a low power 
radio trajismitter 180 and/or a low power radio receiver 182 as shown in the alternate embodiment of 
FIG. 15. The low power radio transmitter transmits the signal for reception by components of the 
production process, and receives signals from the components via the low power radio receiver. The 

30 low power radio transmitter and/or receiver are standard devices in industry. 

FIGS. 16 is an illustration of an exemplary memory medium 184 which can be used with disk 
drives illustrated in FIGS. 14-15. Typically, memory media such as floppy disks, or a CD ROM, or a 
digital video disk will contain, for exan?)le, a multi-byte locale for a single byte language and the 
35 program information for controlling the computer to enable the computer to perform the functions 

described herein. Alternatively, ROM 160 and/or RAM 162 illustrated in FIGS. 14-15 can also be used 
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to store the program informatioQ that is used to instruct the central processing unit 158 to perform the 
operations associated with the instant processes. 

Although computer system 140 is illustrated having a single processor, a single hard disk drive 
5 and a single local memory, the system 140 is optionally suitably equipped with any multitude or 
combination of processors or storage devices. Computer system 140 is, in point of fact, able to be 
replaced by, or combined with, any suitable processing system operative in accordance with the 
principles of the present invention, including sophisticated calculators, and hand-held, laptop/notebook, 
mini, mainframe and super computers, as well as processing system network combinations of the same. 

10 

Conventional processing system architecture is more fiiUy discussed in Computer Organization 
and Architecture , by William Stailings, MacMillan Publishing Co. (3rd ed. 1993); conventional 
processing system network design is more fully discussed in Data Network Design , by Darren L. 
Spohn, McGraw-ffill, Inc. (1993), and conventional data communications are more fully discussed in 

15 Data Comrn imicarinn s Principles , by R.D. Gitlin, J J. Hayes and S^. Weinstain, Plenum Press ( 1992) 
and in The Irwin Handbook of Telecommunications , by James Harry Green, frwin Professional 
Publishing (2nd ed. 1992). Each of the foregoing publications is incorporated herein by reference. 
Alternatively, the hardware configuration is, for example, arranged according to the multiple instruction 
multiple data (MIMD) multiprocessor fonnat for additional computing efficiency. The details of this 

20 form of computer architecture are disclosed in greater detail in, for example, U.S. Patent No. 5, 163, 1 3 1 ; 
Boxer, A., Where Buses Cannot Go, IEEE Spectrum, February 1995, pp. 41-45; and Bairoso, L.A. et 
al., RPM: A Rapid Prototyping Engine for Multiprocessor Systems, IEEE Computer February 1995, pp. 
26-34, all of which are incorporated herein by reference. 

25 In alternate preferred embodiments, the above-identified processor, and, in particular, CPU 

158, may be replaced by or combined with any other suitable processing circuits, including 
programmable logic devices, such as PALs (progranomable array logic) and PLAs (programmable logic 
arrays). DSPs (digital signal processors), FPGAs (field programmable gate arrays), ASICs (application 
specific integrated circuits), VLSIs (very large scale mtegrated circuits) or the Uke. 

30 

FIG. 17 is an illustration of the architecture of the combined Internet, POTS (plain, old, 
telephone service), and ADSL (asymmetric, digital, subscriber line) for use in accordance with the 
principles of the present invention. Furthermore, it is to be understood that the use of the Internet, 
ADSL, and POTS are for exen^lary reasons only and that any suitable communications networic may 
35 be substituted without departing from the principles of the present faivendon. This particular example is 
briefly discussed below. 
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In FIG. 17, to preserve POTS and to prevent a fault in the ADSL equipment 254, 256 from 
compromising analog voice traffic 226, 296 the voice part of the spectrum (the lowest 4 kHz) is 
separated from the rest by a passive filter, called a POTS splitter 258. 260. The rest of the available 
5 bandwidth - - from about 10 kHz to 1 MHz - - carries data at rates up to 6 bits per second for every 
hertz of bandwidth fit>m data equipment 262, 264, and 294. The ADSL equipment 256 then has access 
to a number of destinations including significandy the Internet 220 or other data communications 
networks, and other destinations 270, 272. 

10 To exploit the higher frequencies, ADSL makes use of advanced nxxlulatioo techniques, of 

which the best known is the discrete multitone (DMT) technology. As its name inq)lies, ADSL 
transmits data asymmetrically - - at different rates upstream toward the central office' 252 and 
downstream toward the subscriber 250. 

15 Cable television providers are providing analogous Internet service to PC users over their TV 

cable systems by means of special cable oxxlems. Such modems are capable of transmitting up to 30 
Mh/s over hybrid fiber^coax system, which use fiber to bring signals to a neighborhood and coax to 
distribute it to individual subscribers. 

20 Cable nKxiems come in many forms. Most create a downstream data stream out of one of the 

6-MHz TV channels that occupy spectrum above 50 MHz (and more likely 550 MHz) and carve an 
upstream channel out of the 5-50-MHz band, which is currendy unused. Using 64-state quadrature 
amplitude nxxiulation (64 QAM), a downstream channel can realistically transmit about 30 Mb/s (the 
oft-quoted lower speed of 10 Mb/s refers to PC rates associated with Ethernet coimections). Upstream 

25 rates differ consid^ably from vendor to vendw, but good hybrid fibei/coax systems can deliver 

upstream speeds of a few megabits per second. Thus, like ADSL, cable modems transmit much more 
information downstream than upstream. Then Internet architecture 220 and ADSL architecture 254, 
256 may also be combined with, for exanaple, user networks 222, 224, and 228. 

30 hi accordance with the principles of the present invention, in (xie example, a main computing 

server implementing the process of the invendon may be located on one computing node or terminal 
(e.g., on user network 222, or system 240). Then, various users may interface with the main server via, 
for instance, the ADSL equipment discussed above, and access the information and processes of the 
present invention from remotely located PCs. More specifically, a process 242 capable of performing 

35 the analyst evaluating procedure of the instant invendcm may be advantageously implemented in system 
240. As depicted in FIG. 17, process 242 may access a list of analysts stored, for instance in data 



wo 01/80124 



PCT/USOl/11691 



46 

structure 294. FurthenoMe, although data structure 294 is shown as being stored in system 240, it may 
just as easily be located in a remote and distinct computer system. For example, process 242 may be 
implemented in such a manner as to have access to one of any number of databases 281, 282, 283, 285, 
286, 287 or 288. 

Furthermore, the analyst evaluating process of the present invention may also be inxplemented 
manually. For instance, it is possible to evaluate analysts by hand without the assistance of computing 
systems. 

The many features and advantages of the invention are apparent from the detailed specification, 
and thus, it is intended by the appended claims to cover all such features and advantages of the 
invention which fall within the true spirit and scc^ of the invention. Further, since numerous 
modifications and variations will readily occur to those skilled in the ait, it is not desired to limit the 
invention to the exact construction and operation illustrated and described, and accordingly, all suitable 
modifications and equivalents may be resorted to, falling withia the scope of the invention. 
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CLAIMS 



What is claimed is: 

S 1. A method of evaluating an analyst's performance, said method comprising the steps of: 

utilizing information pertaining to at least one revision made by the analyst involving at least 
one investment, including an upward oc downward change in opinion of the analyst with regard to the 
at least one investment; 

calculating a conditional performance score indicative of the analyst's performance relative to 
10 other analysts, said raw conditional performance score determined at least in part by considering a 
measure of variability of the analyst's performance, an average historical performance of the at least 
one investment following the at least one revision, a standard deviation of the at least one revision, a 
number of revisions made by the analyst, and a likelihood that the at least one revision will actually 
produce an expected result; and 
15 adjusting said conditional performance score according to a return amount adjustment to 

produce a final performance score. 

2. The method of claim 1, wherein the average historical performance of the at least one 
investment is considered over three distinct periods of time. 

20 

3. The method of claim 2, wherein said three distinct periods of tinae comprise a short period of 
time which is approximately five days, an mtermediate period of time which is approximately twenty 
days, and a long period of time which is approximately sixty days. 

25 4. The method of claim 1, wherein the final performance score increases with the number of 
revisions. 



30 



5. The method of claim 1, wherein said revision comprises one of an upward revision or a 
downward revision. 

6. The method of claim 1, wherein the final performance score increases when a rate of return of 
the at least one investment increases and the revision is an upward revision, or when a rate of return of 
the at least one investment decreases and the revision is a downward revision. 
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7. The method of claim 1, wherein the final perfonnance score decreases when a rate of return of 
the at least one investment decreases and the revision is an upward revision, or when a rate of return of 
the at least one investment increases and the revision is a downward revision. 

S 8. The method of claim 1, further comprising adjusting said conditional perfonnance score 
according to a modifier accentuating the number of revisicms made by the analyst 

9. The method of claim 1, wherein said return amount adjustment comprises a small return / 
penalty reducing said final performance score for returns on said at least oae investment less than a 

10 predetermined amount. 

10. The method of claim 9, wherein said small return penalty is conq}rised of a difference between 
an average of a short period of time return factcx*, an intermediate period of time retom factor, and a 
long period of time return factor, and a penalty; wherein said penalty is equal to approximately 15% for 

IS upward revisions and approximately -8% for downward revisions minus a return value corresponding 
to the investment for said long period of time; and wherein each period of tune return factor is a cube 
root of a return percentage over a corresponding period of time. 

1 1 . The method of claim 9, wherein said small return penalty is imposed for small returns over a 
20 long period of time of approximately sixty days after said at least one revision. 

12. The method of claim 9, wherein said small return penalty is generated by subtracting a penalty 
from an average of a first, second and thitd period of time return factor; wherein for upward revisions 
said penalty is 0 if an average return value corresponding to the investment for a long term period of 

25 time is greater than 15%, else said penalty is equal to 15% minus said average return value 

corresponding to the investment for a long term period of time; wherein for downward revisions said 
penalty is 0 if said average return value corresponding to the investment for a long term period of time 
Is less (ban -8%, else said penalty is equal to -8% minus said average return value corresponding to the 
investment for a long term period of time; wherein said first period of time return factor is ten if an 

30 average return over said first period of time is greater than ten, else said first period of time return 

factor is equal to a cube root of a product of 100 and said average return over said first period of time; 
wherein said second period of time return factor is ten if an average return over said second period of 
time is greater than ten, else said second period of time return factor is equal to a cube root of a product 
of 100 and said average return over said second period of time; and wherebi said third period of tinae 

35 return factor is ten if an average return over said third period of time is greater than ten, else said third 
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period of time return fector is equal to a cube root of a product of 100 and said average return over said 
third period of tiine. 

13. The method of claim 1, wherein said calculating a conditional performance score comprises: 
5 detennining a first average return value (ave. retumi), second average return value (ave, 

retum2), and third average return value (ave. retuma) for the at least one revision, said first, second, and 
third average return values corresponding respectively to an average rate of return on the at least one 
revision involving the investnient for a first period of time (timeO, a second period of time (timej, and 
a third period of time (timea); 

10 calculating, in absolute terms, a first t-stat value (t-stati) for said first average return value, a 

second t-stat value (t-stat2) for said second average return value, and a third t-stat value (t-stat3) for said 
third average return value, said first, second, and third t-stat values calculated by utilizing said first, 
second, and third average return values, a first standard deviation (SD]), a second standard deviation 
(SD2), and a third standard deviaticni (SD3), a total niunber of revisions (revisionsO occurring more than 

15 or equal to the first period of time before said evaluating, a total number of revisions (revisions2) 

occurring uKm than or equal to the second period of time before said evaluating, and a total number of 
revisions (rcvisionss) occurring more than or equal to the third period of time before said evaluating; 

calculatiQg a first degree of fteedom (DFO for said first average return value, a second degree 
of freedom (DF2) for said second average return value, and a third degree of fi^edom (DF3) for said 

20 third average return value; 

determining a fixst probability (probi) for said first period of time by utilizing said first t-stat 
value and said first degree of freedom, a second probability (probz) for said second period of time by 
utilizing said second t-stat value and said second degree of freedom, and a third probability (probs) for 
said third period of tin^ by utilizing said third t-stat value and said diird degree of freedom; 

25 generating a raw conditional performance score fw the analyst by utilizing probi , prob2, and 

probs; 

generating an adjustment to said raw conditional performance score by utilizing revisionsi, 
revisions2, and revisionss; and 

adding said adjustment to said raw conditional performance score to produce said conditional 
30 performance score. 

14. The method of claim 1 3, wherein if SDi = 0, then ccmditional performance score = 0 otherwise 
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Tstatx = 



ove. return 1 



f SDl 
^revisioTisl 



; wherein if SD2, = 0, then conditional perfonnance score = 0 otherwise 

ave, retumZ 



Tstat2 



SDl 



-J revisions! 



; wherein if SD3, = 0, then conditional perfonnance score = 0 otherwise 

ove. retumS 



Tstati = 



f SD3 ^ 



^^|revisions3 

; wherein DFi = revisionsi - 1; DF2= revisions2 - 1; and DF3 = revisionsa - 1 

15. The method of claim 13, wherein said probabilities are determined according to a 
predetermined look-up table using actual calculated values of: DFi, DF2, DF3, t-stati, t-stat2, and t-stats, 
10 unless: DFi > 100, in which case DFi is set to 100; DF2 > 100, m which case DF2 is set to 100; DF3 > 
100, in which case DF3 is set to 100; t-stati > 20, in which case t-stati is set to 20; t-stat2 > 20, m which 
case t-stat2 is set to 20; t-stat3 > 20, in which case t-stat3 is set to 20; t-stati < 0.01, in which case probi is 
set to 0; t-stat2 < 0.01, in which case prob2 is set to 0; or t-sUt, < 0.01, in which case probs is set to 0. 



15 16. The method of claim 13, wherein 

[ [1 - (J + or - probl)] + [l - (.5 + or - prob2)h [l " (-5 + or - prob3)\ 



cond _ performance _ score 



•"lOO 



wherein for upward revisions probi is subtracted from 0.5 if ave, retumi > 0, else probi is added to 0.5 
wherein for upward revisions prob2 is subtracted from 0.5 if ave. retumi > 0, else probz is added to 0.5 
wherein for upward revisions prob3 is subtracted from 0.5 if ave. returns > 0, else probs is added to 0.5 
20 wherein for downward revisions probi is subtracted from 0.5 if ave. retumi < 0, else probi is added to 
0.5; wherein for downward revisions prob2 is subtracted from 0.5 if ave. retum2 < 0, else proba is added 
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to 0.5; and wherein for downward revisions proba is subtracted from 03 if ave. retuma < 0, else probs is 
added to 0.5. 

17. The mediodofclaim 13, wherein 



18- A method of evaluating an analyst's performance, said method comprising the steps of; 

utilizing information pertaining to at least one upward or downward revision including an 
upward or downward change in opinion of the analyst with regard to the at least one investment; 
calculating a performance scgcq indicative of the analyst's perfonnance relative to other 

10 analysts, said performance score determined at least in part by considering a measure of variability of 
the analyst's performance, an average historical performance of the at least one investment following 
the at least one upward or downward revision, a standard deviation of the at least one upward or 
downward revision, a number of upward or downward revisions made by the analyst, and a likelihood 
that the at least one upward or downward revision will actually produce an expected result; and 

15 comparing the analyst's performance score against performance scores of other analysts to 

produce at least one of an upward, downward or conobined revision ranking. 

19. The method of claim 18, wherein said three distinct periods of time comprise a short period of 
time which is approximately five days, an intermediate period of time which is approximately twenty 

20 days, and a long period of time which is approximately sixty days. 

20. The method of claim 18, wherein said performance score increases with the number of 
revisions. 

25 21. The method of claim 18, further comprising adjusting said performance score according to a 
modifier accentuating the number of revisions made by the analyst 

22. The method of claim 18, wherein said performance score increases when a rate of return of the 
at least one investnent increases and the revision is an upward revision, or when a rate of return of the 
30 at least one investment decreases and the revision is a downward revision, and wherein said 

performance score decreases when a rate of return of the at least one investment decreases and the 
revision is an upward revision, or when a rate of return of the at least one investment increases and the 
revision is a downward revision. 



1 





5 
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23. The method of claim 1 8, further comprising adjusting said performance score according to a 
small return penalty. 

5 24. The method of claim 23, wherein said small return penalty is generated by subtracting a penalty 
from an average of a first, second and third period of time return factor, wherein for upward revisions 
said penalty is 0 if an average return value conesponding to the investment for a long term period of 
time is greater than 15%, else said penalty is equal to 15% minus said average return value 
corresponding to the investment for a long term period of time; wherein for downward revisions said 

10 penalty is 0 if said average return value corresponding to the investment for a long term period of time 
is less than -8%, else said penalty is equal to -8% minus said average return value corresponding to the 
investment for a long term period of tinae; wherein said first period of time return factor is ten if an 
average return over said first period of time is greater than ten, else said first period of time return 
factor is equal to a cube root of a product of 100 and said average return over said first period of time; 

15 wherein said second period of time return factor is ten if an average return over said second period of 
time is greater than ten, else said second period of time return factor is equal to a cube root of a product 
of 100 and said average return over said second period of time; and wherein said third period of time 
return factor is ten if an average return over said third period of time is greater than tai, else said third 
period of time return factor is equal to a cube root of a product of 100 and said average return over said 

20 third period of tinoe. 

25. The HKthod of claim 18, wherein said calculating a performance score comprises: 

determining a first average return value (ave. retumi), second average return value (ave. 
retumj). and third average return value (ave. returns) for the at least one revision, said first, second, and 

25 third average return values corresponding respectively to an average rate of return on the at least one 
revision involving the investnoent for a first period of time (timeO, a second period of time (timez), and 
a third period of time (dbne?); 

calculating, in absolute terms, a first t-stat value (t-stati) for said first average return value, a 
second t-stat value (t-stat2) for said second average return value, and a third t-stat value (t-stata) for said 

30 third average return value, said first, second, and third t-stat values calculated by utilizing said first, 
second, and third average return values, a first standard deviation (SDi), a second standard deviation 
{SD2), and a third standard deviation (SD3), a total number of revisions (revisionsi) occurring more than 
or equal to the first period of time before said evaluating, a total number of revisions (revisionsj) 
occurring more than or equal to the second period of time before said evaluatiug, and a total number of 

35 revisicMis (revisionsa) occurring more than or equal to the third period of time before said evaluating; 
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calculating a first degree of bccdom (DFi) fcx- said first average retuTD value, a second degree 
of freedom (DFi) for said second average return value, and a third degree of freedom (DF3) for said 
third average return value; 

determining a first probability (probi) for said first period of time by utilizing said first t-stat 
value and said first degree of freedom, a second probability (probz) for said second period of time by 
utilizing said second t-stat value and said second degree of freedom, and a third probability (probs) for 
said third period of time by utilizing said third t-stat value and said third degree of freedom; and 

generating a conditional performance score (cond. performance score) for the analyst by 
utilizing probi, probz, and probs,, which may be manipulated to produce said performance score. 

26. The method of claim 25, wherein if SDj = 0, then conditional performance score = 0 otherwise 

ovc retuml 



Tstatx = 



SD\ 



V revisions! 



; wherein if SD2, = 0, then conditional performance score = 0 otherwise 

ave. retumZ 



T stall = 



SD2 



-previsions! 



15 ; wherein if SD3, = 0, then conditional performance score = 0 otherwise 

ave. returns 



Tstatx = 



SD3 



V revisions3 



; wherein DFi = revisionsi - 1; DF2 = revisionsj - 1; and DF3 = revisionsa - 1 



27. The method of claim 25, wherein said probabilities are determined according to a 
20 predetermined look-up table using actual calculated values of: DFi, DF2, DF3, t-stati, t-stat2, and t-stata, 
unless: DFi > 100, in which case DFi is set to 100; DF2 > 100, in which case DF2 is set to 100; DF3 > 
100, in which case DF3 is set to 100; t-stati > 20, in which case t-stati is set to 20; t-stat2> 20, m which 
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case t-stat2is set to 20; t-stata > 20, in which case t-stats is set to 20; t-stati < 0.01, in which case probi is 
set to 0; t-stat2 < 0.0 1 , in which case prob2 is set to 0; or t-stata < 0.0 1 , in which case prob3 is set to 0. 



28. The nzethod of claim 25, wherein 



cond _ performance _ score 



^ I [l - (.5 + or - probl)] -f [l - (.5 + or - profc2)]+ [l - (.5 + or - probH)] 



wherein for upward revisions probi is subtracted from 0.5 if ave. retumi > 0, else probi is added to 0.5 
wherein for upward revisions probz is subtracted from 0.5 if ave. retumi > 0, else prob2 is added to 0.5 
wherein for upward revisions probs is subtracted from 0.5 if ave. returns > 0, else probs is added to 0.5 
wherein fw downward revisions probi is subtracted from 0.5 if ave. retumi < 0, else probi is added to 
10 0.5; wherein for downward revisions prob2 is subtracted from 0.5 if ave. retunh < 0, else prob2 is added 
to 0.5; and wherem for downward revisions proba is subtracted from 0.5 if ave. returns < 0, else probs is 
added to 0.5. 



29. A method of evaluating an analyst's perfcwmance by utilizing information pertaining to at least 
15 one revision made by the analyst involving at least one investnoent, said ruethod comprising the steps 
of: 

determining a first average return value (ave. retumO, second average return value (ave. 
retum2), and third average return value (ave. returns) for the at least one revision, said first, second, and 
third average rctum values corresponding respectively to an average rate of retum on the at least one 
20 revision involving the investment for a first period of time (timeO, a second period of time (time2). and 
a third period of time (times); 

calculating, in absolute terms, a first t-stat value (t-stati) for ave. retumi, a second t-stat value 
(t-stat2) fcff ave. retum2, and a third t-stat value (t-stats) for ave. returns, said first, second, and third t- 
stat values calculated by utilizing ave. retumi, ave. retum2, ave. returns, a first standard deviation (SDj), 
25 a second standard deviation (SD2), and a third standard deviation (SD3), a total numb^ of revisions 
(revisionsi) occuning more than or equal to timei before said evaluating, a total number of revisions 
(revisions2) occurring more than or 6qual to time2 before said evaluating, and a total number of 
revisions (revisionss) occurring more than or equal to tinacs before said evaluating; 

calculating a first degree of freedom (DFi) for ave. retumi, a second degree of freedom (DF2) 
30 for ave. retumi, and a third degree of freedom (DF3) for ave. rctums; 

determining a first probability (probi) for timei by utilizing t-stati and DP), a second probability 
(prob2) fcM" timei by utilizing t-stati and DF2, and a third probability (probs) for times by utilizmg t-stats 
andDFs; 

generating a raw conditional performance score for the analyst by utilizing probi, probi, probs; 
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adding a modifier (adjustment) to the raw conditional performance score determined by 
utilizing revisionsi, revisioiiS2, and revisionsa, thereby producing a conditional performance score 
(cond. performance score); . 

generating a small return penalty by subtracting a penalty fi'om an average of return factors 
5 corresponding to timei . timeo, and times, wherein for upward revisions said penalty is 0 if ave. returns is 
greater than 15%, else said penalty is equal to 15% minus ave. retuma, wherein for downward revisions 
said penalty is 0 if ave. returns is less than -8%. else said penalty is equal to -8% minus ave. returns, 
wherein said first period of time return factor is ten if ave. re turn i is greats than ten, else said first 
period of time return factor is equal to a cube root of a product of 100 and ave. retumi, wherein said 
10 second period of tinae return factor is ten if ave. returuz is greater than ten, else said secc»Kl period of 
time return factor is equal to a cube root of a product of 100 and ave. retum^, and wherein said third 
period of time return factor is ten if ave. returns is greater than ten, else said third period of time return 
factor is equal to a cube root of a product of 100 and ave. returns; and 

generating a final performance score (perfcxmance score) indicative of said analyst's 
15 performance by adding said small return penalty to said conditional performance score. 

30. A system for evaluating an analyst's perfonnance, said system comprising: 
a processor, 

a memory storing a computer program controlling operation of said processes, said program 
20 including instructions for causing the processor to effect 

utilizing information pertaining to at least one revision made by the analyst involving at least 
one investment, including an upward or downward change in opinion of the analyst with regard to the 
at least one investment; 

calculating a conditional performance score indicative of die analyst's performance relative to 
25 otbex analysts, said raw conditional performance score determined at least in part by ccHisidering a 
measure of variability of the analyst's perfmnance, an average historical performance of the at least 
one investment following the at least one revision, a standard deviation of the at least one revision, a 
number of revisions made by the analyst, and a likelihood that the at least one revision will actually 
produce an expected result; and 
30 adjusting said conditional perfiormance score according to a return amount adjustment to 

produce a final perfonnance score. 

31. The system of claim 30, wherein said average historical performance is calculated over three 
distinct periods of time, which comprise a short period of time which is approximately five days, an 

35 intermediate period of time which is approximately twenty days, and a long period of time which is 
approximately sixty days. 
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32. The system of claim 30, wherein the final perfonnance score increases when a rate of return of 
the at least one mvestment increases and the revision is an upward revision, or when a rate of return of 
the at least oi>e investment decreases and the revision is a downward revision, and wherein the final 

5 performance score decreases when a rate of return of the at least one investment decreases and the 
revision is an upward revision, or when a rate of return of the at least one investment increases and the 
revision is a downward revision. 

33. The system of claim 30, wherein said program further comprises instructions to effect adjusting 
10 said conditional performance score according to a modifier accentuating the number of revisions made 

by the analyst 

34. The system of claim 30, wherein said return amount adjustment conq)rises a small return 
penalty reducing said fiTial performance score for returns on said at least one investment less than a 

15 predetermined amount 

35. The system of claim 34, wherein said small return penalty is comprised of a difference between 
an average of a short period of time return factor, an intermediate period of tinoe return factor, and a 
long period of time return factor, and a penalty; wherein said penalty is equal to approximately 15% for 

20 upward revisions and approximately —8% for downward revisions minus a return value corresponding 
to the mvestment fw said long period of time; and wherein each period of time return factor is a cube 
root of a return percentage over a corresponding period of time. 

36. The system of claim 34, wherein said small return penalty is imposed for small returns over a 
25 long period of time of approximately sixty days after said at least one revision. 

37. The system of claim 34, wherein said small return penalty is generated by subtracting a penalty 
from an average of a first, second and third period of time return factor; wherein for upward revisions 
said penalty is 0 if an average return value corresponding to the investment for a long term period of 

30 time is greater than 15%, else said penalty is equal to 15% minus said average return value 

corresponding to the investment for a long term period of time; wherein for downward revisions said 
penalty is 0 if said average return value corresponding to the investment for a long term period of time 
is less than -8%, else said penalty is equal to -8% minus said average return value corresprading to the 
investment for a long term period of time; wh^ein said first period of time return factor is ten if an 

35 average return over said first period of time is greater than ten, else said first period of time return 

factor is equal to a cube root of a product of 100 and said average return over said first period of time; 
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wherein said second period of time return factor is ten if an average return over said second period of 
time is greater than tea, else said second period of time return factcxr is equal to a cube root of a product 
of 100 and said average return over said second period of time; and wherein said third period of time 
return factor is ten if an average return over said third period of time is greater than ten, else said third 
5 period of time return factor is equal to a cube root of a product of 100 and said average return over said 
third period of time. 

3. A system for evaluating an analyst's performance, said system comprising: 

a processor, 

10 a memory storing a computer program controlling operation of said processor, 
said program including instructions for causing the processor to effect: 

utilizing information pertaining to at least one upward or downward revision including an 
upward or downward change in <:q>inion of the analyst with regard to the at least one investment; 
calculating a performance score indicative of the analyst's perfonnance relative to other 

IS analysts, said performance score determined at least in part by considering a measure of variability of 
the analyst's performance, an average historical performance of the at least one investment following 
the at least one upward oc downward revision, a standard deviation of the at least one upward or 
downward revision, a number of upward or downward revisicms made by the analyst, and a likelihood 
that the at least one upward or downward revision will actually produce an expected result; and 

20 comparing the analyst's performance score against performance scores of other analysts to 

produce at least one of an upward, downward or combined revision ranking. 

39. The system of claim 38, wherein said program further comprises instructions to effect adjusting 
said perfonnance score according to a modifier accentuating the number of revisions made by the 

25 analyst 

40. The system of claim 38, wherein said perfcsmance score increases when a rate of return of the 
at least one investment increases and the revision is an upward revision, or when a rate of return of the 
at least one investnoent decreases an^ the revision is a downward revision, and wherein said 

30 performance score decreases when a rate of retnm of the at least one investment decreases and the 

revision is an upward revision, or when a rate of return of the at least cxie investment increases and the 
revisicm is a downward revision. 

4 1 . The system of claim 38, wherein said calculating a performance score conqmses: 

35 determining a first average return value (ave. retumO, second average return value (ave. 

retunh), and tiurd average return value (ave. returns) for the at least one revision, said first, second, and 
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third average return values conesponding respectively to an average rate of return on the at least one 
revision involving the investment for a first period of time (timei), a second period of time (time2)» and 
a third period of time (times); 

calculating, in absolute terms, a first t-stat value (t-stati) for said first average return value, a 
second t-stat value (t-stati) for said second average return value, and a third t-stat value (t-stats) for said 
third average return value, said first, second, and third t-stat values calculated by utilizing said first, 
second, and third average return values, a first standard deviation (SDi), a second standard deviation 
(SDi), and a third standard deviation (SD3), a total number of revisions (revisionsO occurring OKjre than 
or equal to the first period of time before said evaluating, a total number of revisions (revisions2) 
occurring more than or equal to the second period of time before said evaluating, and a total number of 
revisions (revisionsa) occurring more than or equal to the third period of time before said evaluating; 

calculating a first degree of freedom (DFO for said first average return value, a second degree 
of fireedom (DF2) for said second average return value, and a third degree of freedom (DFs) for said 
third average return value; 

detennining a first probability (prob^) for said first period of time by utilizing said first t-stat 
value and said first degree of freedom, a second probability (probz) for said second period of time by 
utilizing said second t-stat value and said second degree of freedom, and a third probability (proba) for 
said third period of time by utilizing said third t-stat value and said third degree of freedom; and 

generating a conditional performance score (cond. performance score) for the analyst by 
utilizing probi, probz, and probs,, which may be manipulated to produce said performance sccffe. 



42. The system of claim 4 1 , wherein if SDi = 0, then conditional perf ormance score = 0 otherwise 

ove. retuml 



T stats 



SDI 



V revisionsl 



; wherein if SD2, = 0, then conditional performance score = 0 otherwise 

ave, return! 



Tstat2 = 



SD2 



^J revisions! 



; wherein if SD3, = 0, then conditional performance score = 0 otherwise 

ave. retumS 
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Tstatz = 



f 



SD3 



revisions3 



J 



; wherein DFi = revisic«iSi - 1; DF2= revisioiiS2 - 1; and DF3 = revisiooss - 1 

43. The system of claim 41, wherein said probabilities are determined according to a predetermined 
look-up table using actual calculated values of: DFi, DF2, DF3, t-stati, t-stat2, and t-sUt,, unless; DFi > 
100, m which case DFi is set to 100; DF2 > 100, in which case DF2 is set to 100; DF3 > 100, in which 
case DF3 is set to 100; t-stati > 20, in which case t-stati is set to 20; t-stat2 > 20, in which case t-stat2 is 
set to 20; t-stats > 20, in which case t-stata is set to 20; t-stati < 0.01, in which case probi is set to 0; t- 
stat2 < 0.01, in which case prob2 is set to 0; or t-stata < 0 01 , in which case proba is set to 0. 

44. The system of claim 4 1 , wherein 



wherein for upward revisions probi is subtracted fiom 0.5 if ave. retumi > 0, else probi is added to 0.5; 
wherein for upward revisions probz is subtracted from 0.5 if ave. retum2 > 0, else prob2 is added to 0.5; 
wherein for upward revisions proba is subtracted from 0.5 if ave. returua > 0, else proba is added to 0.5; 
wherein for downward revisions probi is subtracted from 0.5 if ave. retumi < 0, else probi is added to 
0.5; wherein for downward revisions probz is subtracted from 0.5 if ave. retum2 < 0, else prob2 is added 
to 0.5; and wherein for downward revisions proba is subtracted from 0.5 if ave. retuma < 0, else proba is 
added to 0.5. 

45. A system for evaluating an analyst's performance, said system comprising: 

means for utilizing information potaining to at least one revision made by the analyst involving 
at least one investment, including an upward or downward change in opinion of the analyst with regard 
to the at least one investment; 

means for calculating a conditional performance score indicative of the analyst's performance 
relative to othCT aiudysts, said raw conditional performance score determined at least in part by 
considering a measure of variability of the analyst's performance, an average historical performance of 
the at least one investment following the at least one revision, a standard deviation of the at least one 
revision, a number of revisions made by the analyst, and a likelihood that the at least one revision wiU 
actually produce an expected result; and 



cond _ performance _ score — 



- (.5 + or - proM)] + [l - (.5 + or - pn?£>2)]+ [l - ( J + or - pro\ 



3 
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means for adjusting said conditional performance score according to a return amount 
adjustment to produce a final perfcHinance score. 

46. The system of claim 45, wherein said average historical performance is calculated over three 
5 distinct periods of time, which comprise a short period of time which is approximately five days, an 

interaiediate period of time which is approximately twenty days, and a long period of time which is 
approximately sixty days. 

47. The system of claim 45, wherein the final performance score increases when a rate of return of 
10 the at least one investment increases and the revision is an upward revision, or when a rate of return of 

the at least one investment decreases and the revision is a downward revision, and wherein the final 
performance score decreases when a rate of return of the at least one investment decreases and the 
revision is an upward revision, or when a rate of return of the at least one investment increases and the 
revision is a downward revisioiL 

15 

48. The system of claim 45. further conq>rising means for adjusting said conditional performance 
score according to a nKxlifiCT accentuating the number of revisions made by the analyst 

49. The system of claim 45, wherein said return amount adjustment comprises a small return 
20 penalty reducing said final performance score for returns on said at least one mvestment less than a 

predetermined amount. 

50. The system of claim 49, wherein said small return penalty is comprised of a difference between 
an average of a short period of time return factor, an intermediate period of time return factor, and a 

25 long period of time return factor, and a penalty; wherein said penalty is equal to approximately 15% for 
upward revisions and approximately -8% for downward revisions minus a return value corresponding 
to the investment for said long period of time; and wherein each period of time return factor is a cube 
root of a return percentage over a corresponding period of time. 

30 51. The system of claim 49, wherein said small return penalty is imposed for small returns over a 
long period of time of approximately sixty days after said at least one revision. 

52. The system of claim 49. wherein said small return penalty is generated by subtracting a penalty 
from an average of a first, second and third period of time return factor, wherein for upward revisions 
35 said penalty is 0 if an average return value corresponding to the investment for a long term period of 
time is greater than 15%, else said penalty is equal to 15% minus said average return value 
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conespoading to the investment for a long term period of time; wherein for downward revisions said 
penalty is 0 if said average return value corresponding to the investment for a long term period of tiroe 
is less than -8%» else said penalty is equal to -8% minus said average return value corresponding to the 
investment for a long term period of time; wherein said first period of time return factor is ten if an 
5 average return over said first period of time is greater than ten, else said first period of time return 
factor is equal to a cube root of a product of 100 and said average return over said first period of time; 
wherein said second period of time return factor is ten if an average return over said second period of 
time is greater than ten, else said second period of tinae return factor is equal to a cube root of a product 
of 100 and said average return over said seccxid period of time; and wherein said third period of time 
10 return factor is ten if an average return over said third period of time is greater than ten, else said third 
period of time return factor is equal to a cube root of a product of 100 and said average return over said 
third period of time. 

53. A system for evaluating an analyst's perfonnance, said system comprising: 

15 means for utilizing information pertainmg to at least one upward or downward revision 

including an upward or downward change in opinion of the analyst with regard to the at least one 
investment; 

means for calculating a performance score indicative of the analyst's performance relative to 
other analysts, S£ud performance score determined at least in part by considering a measure of 
20 variability of the analyst's performance, an average historical performance of the at least one 

investment following the at least one upward or downward revision, a standard deviation of the at least 
one upward or downward revision, a number of upward or downward revisions made by the analyst, 
and a likelihood that the at least one upward or downward revision will actually produce an expected 
result; and 

25 means fc^* comparing the analyst's performance score against performance scores of other 

analysts to produce at least one of an upward, downward or combined revision ranking. 

54. The system of claim 53, further comprising means for adjusting said performance score 
according to a modifier accentuating the number of revisions made by the analyst 

30 

55. The system of claim 53, wherein said perfcnmance score increases when a rate of return of the 
at least one investment increases and the revision is an upward revision, or when a rate of return of the 
at least one investment decreases and the revision is a downward revision, and wherein said 
performance score decreases when a rate of return of the at least one investment decreases and the 

35 revision is an upward revision, or when a rate of return of the at least one investment increases and the 
revision is a downward revision. 
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56. The system of claim 53, wherein said calculating a perfonnance score conqprises: 

detemdning a first average return value (ave. retumi), second average return value (ave. 
returai), and third average return value (ave. returns) for the at least one revision, said first, second, and 
third average return values corresponding respectively to an average rate of return on the at least one 
revision involving the investment for a first period of time (timei), a second period of time (time2), and 
a third period of time (timea); 

calculating, in absolute terms, a first t-stat value (t-stati) fcff said first average return value, a 
second t-stat value (t-stat2) for said second average return value, and a third t-stat value (t-stats) for said 
third average return value, said first, second, and third t-stat values calculated by utilizing said first, 
second, and third average return values, a first standard deviation (SDi), a second standard deviation 
(SD2). and a third standard deviation (SD3), a total number of revisions (revisionsO occurring more than 
or equal to the first period of tiroe before said evaluating, a total numb^ of revisions (revisicmsa) 
occurring more than or equal to the second period of time before said evaluating, and a total number of 
revisions (rcvisionsa) occurring more than or equal to the third period of time before said evaluating; 

calculatmg a first degree of freedom (DFO for said first average return value, a second degree 
of freedom (DF2) for said second average return value, and a third degree of fiieedom (DF3) for said 
third average return value; 

determining a first probability (probi) for said first period of time by utilizing said first t-stat 
value and said first degree of fireedom, a second probability (probi) for said second period of time by 
utilizing said second t-stat value and said second degree of fteedom, and a third probability (probs) for 
said third period of time by utilizing said third t-stat value and said third degree of fireedom; and 

generating a conditional performance score (cond. performance score) for the analyst by 
utilizing probi, prob2, and probs., which may be manipulated to produce said performance score. 



57. The system of claim 56, wherein if SDi = 0, then conditional perfcwmance score = 0 otherwise 

asfe, retuml 



Tstatx 



SDI ^ 



V revisionsl 



; wherein if SD2, = 0, then conditional performance score = 0 otherwise 

ave. retum2 

Tstatz = 
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SD2 



previsions! 



; wherein if SD3, == 0, then conditional performance score = 0 otherwise 



ave. returns 



Tstat^ = 



SD3 




J 



; wherein DFi= revisionsi - 1; DF2= rcvisions2 - I; and DF3 = revisionss - 1 

5 

58. The system of claim 56, wherein said probabilities are determined according to a predetermined 
look-up table using actual calculated values oft DFi, DF2, DF3, t-stati, t-stat2, and t-stata, unless: DFi > 
100, m which case DFi is set to 100; DF2 > 1(X), in which case DF2 is set to 100; DF3 > 100, in which 
case DF3 is set to 100; t-stati > 20, in which case t-stati is set to 20; t-stat2 > 20, in which case t-stat2 is 

10 set to 20; t-stat3> 20, in which case t-stats is set to 20; t-stati < 0.01, in which case probi is set to 0; t- 
stat2< 0.01, in which case prob2 is set to 0; or t-stats < 0.01, in which case prob3 is set to 0. 

59. The system of claim 56, wherein 



15 wherein for upward revisions probi is subtracted from 0.5 if ave. retumi > 0, else probt is added to 0.5; 

wherein for upward revisions prob2 is subtracted from 0.5 if ave. Tetum2 > 0, else prob2 is added to 0.5; 

wherein fw upward revisions prob3 is subtracted from 0.5 if ave. rctum3 > 0, else probs is added to 0.5; 

wherein fcH* downward revisions probi is subtracted from 0.5 if ave. retumj < 0, else probi is added to 

0.5; wherein for downward revisions prob2 is subtracted from 0.5 if ave. retum2 < 0, else prolh is added 
20 to 0.5; and wherein for downward revisions prob3 is subtracted fit>m 0.5 if ave. returua < 0, else proba is 

added to 0.5. 

60. A computer data signal embodied in a carrier wave, said signal bearing instructions for causing 
a computer system to evaluate an analyst's performance, said instructions comprising: 
25 utilizmg information pertaining to at least one revision made by the analyst involving at least 

one investment, including an upward or downward change in opinion of the analyst with regard to the 
at least one investcoent; 



cond _ perfontumce _ score = 



- (.5 + or - probi)] + [l - (.5 A- or- pro62)]+ [l - (.5 + or - prob3)] 
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calculating a conditioQai performance score indicative of the analyst's perfonnance relative to 
othCT analysts, said raw conditional performance score determined at least in part by considering a 
measure of variability of the analyst's performance, an average historical performance of the at least 
one investDoent following the at least one revision, a standard deviation of the at least one revision, a 
5 number of revisions made by the analyst, and a likelihood that the at least one revision will actually 
produce an expected result; and 

adjusting said conditional performance score according to a return amount adjustment to 
produce a final perfonnance score. 

10 61. The signal of claim 60, wherein the final performance score increases when a rate of return of 
the at least one investment increases and the revision is an upward revisicxi, or when a rate of return of 
the at least one investment decreases and the revision is a downward revision, and wherein the final 
performance score decreases when a rate of return of the at least one investment decreases and the 
revision is an upward revision, or when a rate of return of the at least one investment increases and the 

15 revision is a downward revision. 

62. The signal of claim 60, wherein said return amount adjustment comprises a small return penalty 
reducing said final performance score for returns on said at least one investment less than a 
predetermined amount. 

20 

63. The signal of claim 62, wherein said small return penalty is comprised of a difference between 
an average of a short period of time return factor, an intermediate period of time return factor, and a 
long period of time return factor, and a penalty; wherein said penalty is equal to approximately 15% for 
upward revisions and approximately -8% for downward revisicms minus a return value corresponding 

25 to the investment for said long period of time; and wherein each period of time return factor is a cube 
root of a return percentage over a corresponding period of time. 

64. The signal of claim 62, wherein said small return penalty is generated by subtracting a penalty 
from an average of a first, second and third period of time return factor, wherein for upward revisions 

30 said penalty is 0 if an average return value corresponding to the investment for a long term period of 
time is greater than 15%, else said penalty is equal to 15% minus said average return value 
corresponding to the iavestment for a long term period of time; wherein for downward revisions said 
penalty is 0 if said average return value corresponding to the investment for a long term period of time 
is less than -8%, else said penalty is equal to -8% minus said average return value corresponding to the 

35 investment for a long term period of time; wherein said first period of time return factor is ten if an 
average return over said first period of time is greater than ten, else said first period of time return 
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factor is equal to a cube root of a product of 100 and said average return over said first period of time; 
wherein said second period of time return factor is ten if an average return over said second period of 
time is greater than ten, else said second period of time return factor is equal to a cube root of a product 
of 100 and said average return over said second period of time; and wherein said third period of time 
5 return factor is ten if an average return over said third period of time is greater than ten, else said third 
period of time return factor is equal to a cube root of a product of 100 and said average return over said 
third period of time. 

65. A computer data signal embodied in a carrier wave, said signal bearing instructions for causing 
10 a computer system to evaluate an analyst's performance, said instrucdons comprising: 

utilizing information pertaining to at least one upward or downward revision including an 
upward or downward change in opinion of the analyst with regard to the at least one investment; 

calculating a performance score indicative of the analyst's performance relative to other 
analysts, said performance score determined at least in part by considering a measure of variability of 
15 the analyst's performance, an average historical performance of the at least one investment following 
the at least one upward or downward revision, a standard deviation of the at least one upward or 
downward revision, a number of upward or downward revisions made by the analyst, and a likelihood 
that the at least one upward or downward revision will actually produce an expected result; and 

comparing the analyst's performance score against performance scores of other analysts to 
20 produce at least one of an upward, downward or combined revision ranking. 

66. The signal of claim 65, further comprising instructions for adjusting said performance score 
acccHtling to a noodifier accentuating the number of revisions made by the analyst 

25 67. The signal of claim 65, wherein said calculating a performance score comprises: 

determining a first average return value (ave. retumO, second average return value (ave. 
retumi), and third average return value (ave. returns) for the at least one revision, said first, second, and 
third average return values corresponding respectively to an average rate of return on the at least one 
revision involving the investment for a first period of time (timci), a second period of time (tiine2), and 

30 a third period of time (timca); 

calculating, in absolute terms, a first t-stat value (t-stati) for said first average return value, a 
second t-stat value (t-stat2) for said second average return value, and a third t-stat value (t-stats) for said 
third average return value, said first, second, and third t-stat values calculated by utilizing said first, 
second, and third average return values, a first standard deviation (SDi), a second standard deviation 

35 (SD2), and a third standard deviation (SD3), a total number of revisions (revisioosi) occurring more than 
or equal to the first period of time before said evaluating, a total number of revisions (revisions2) 
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occurring more than or equal to the second period of time befort said evaluating, and a total number of 
revisions (levisionsa) occurring more than or equal to the third period of time before said evaluating; 

calculating a first degree of fineedom (DFO for said first average return value, a second degree 
of freedom (DF2) for said second average return value, and a third degree of freedom (DF3) for said 
5 third average return value; 

determining a first probability (probi) for said first period of time by utilizing said first t-stat 
value and said first degree of freedom, a second probability (probz) for said second period of time by 
utilizing said second t-stat value and said second degree of freedom, and a third probability (probs) for 
said third period of time by utilizing said third t-stat valxie and said third degree of freedom; and 
10 generating a conditional performance score (cond. performance score) for the analyst by 

utilizing probi, prob2, and probs,, which xmy be manipulated to produce said performance score. 

68. A computer readable medium storing instructions executable by a conq)ut^, the instructions for 
instructing the computer to effect evaluating an analyst's performance, said instructions con^yrising: 

15 utilizing information pertaining to at least one revision made by the analyst mvolving at least 

one investment, including an iq)ward or downward change in opinion of the analyst with regard to the 
at least one investment; 

calculating a conditional performance score indicative of the analyst's performance relative to 
other analysts, said raw conditional performance score determined at least in part by considering a 

20 measure of variability of the analyst's perfcxmance, an average historical performance of the at least 
one investment following the at least one revision, a standard deviation of the at least one revision, a 
number of revisions made by the analyst, and a likelihood that the at least one revision will actually 
produce an expected result; and 

adjusting said conditional performance score according to a return amount adjustment to 

25 produce a final performance score. 

69. The computer readable medium of claim 68, wherein the final performance score increases 
when a rate of return of the at least one investnoent increases and the revision is an upward revision, or 
when a rate of return of the at least one investment decreases and the revision is a downward revision, 

30 and wherein the fmal performance score decreases when a rate of return of the at least one investooent 
decreases and the revision is an upward revision, oar when a rate of return of the at least one investment 
increases and the revision is a downward revision. 

70. The computer readable medium of claim 68, wherein said return amount adjustment conoprises 
35 a small return penalty reducing said final performance score for returns on said at least one investment 

less than a predetermined amount. 
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71. The coiUJiUer readable medium of claim 70, wherein said small return penalty is comprised of 
a difference betwera an average of a short period of time return factor, an intermediate period of time 
return factor, and a long period of tune return factor, and a penalty; wherein said penalty is equal to 

5 approximately 15% for upward revisiwis and approximately -8% for downward revisions minus a 
return value corresponding to the investment for said long period of time; and wherein each period of 
time return factor is a cube root of a return percentage over a cotresponding period of time. 

72. The computer readable medium of claim 70, wherein said small return penalty is generated by 
10 subtracting a penalty fix)m an average of a first, second and third period of time return factor; wherein 

for upward revisions said penalty is 0 if an average return value corresponding to the investment for a 
long term period of time is greater than 15%, else said penalty is equal to 15% minus said avwage 
return value corresponding to the investment for a long term period of time; wherein for downward 
revisions said penalty is 0 if said average return value corresponding to the investment for a long term 

15 period of time is less than -8%, else said penalty is equal to -8% minus said average return value 

corresponding to the investment for a long term period of time; wherein said first period of time retum 
factOT is ten if an average return over said first period of time is greater than ten, else said first period of 
time retum factor is equal to a cube root of a product of 100 and said average return over said first 
period of time; wherein said second period of time retum factor is ten if an average retum over said 

20 second period of tintie is greater than ten, else said second period of time retum factor is equal to a cube 
root of a product of 100 and said average retum over said second period of time; and wherein said third 
period of time retum factor is ten if an average retum over said third period of time is greater than ten, 
else said third period of time retum factor is equal to a cube root of a product of 100 and said average 
retum over said third period of time. 

25 

73. A computer readable mediiun storing instmctions executable by a computer, the instructions for 
instructing the computer to effect evaluating an analyst's performance, said instructions conq>rising: 

utilizing ioformatioQ pertaining to at least one upward or downward revision including an 
upward or downward change in opinion of the analyst with regard to the at least one investment; 

30 calculating a performance score indicative of the analyst's perfonnance relative to other 

analysts, said pof c^mance score determined at least in part by considering a measure of variability of 
the analyst's performance, an average historical performance of the at least one investment following 
the at least one upward or downward revision, a standard deviation of the at least one upward or 
downward revision, a number of upward or downward revisions made by the analyst, and a likelihood 

35 that the at least one upward or downward revision will actually produce an expected result; and 
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comparing the analyst's perfonnance score against performance scores of other analysts to 
produce at least one of an upward, downward or combined revision ranking. 

74. The computer readable medium of claim 73, further comprising adjusting said performance 
5 score according to a modifier accentuating the ntunber of revisions made by the analyst 

75. The computer readable medium of claim 73, wherein said calculating a performance score 
comprises: 

determining a first average return value (ave. retumi), second average return value (ave. 

10 retunh), and third average return value (ave, returns) for the at least one revision, said first, second, and 
third average return values corresponding respectively to an average rate of return on the at least one 
revision involving the investment for a fint period of time (timet), a second period of time (timei), and 
a third period of time (timea); 

calculating, in absolute terms, a first t-stat value (t-statO for said first average return value, a 

15 second t-stat value (t-stat2) for said second average return value, and a third t-stat value (t-stata) for said 
third average return value, said first, second, and third t-stat values calculated by utilizing said first, 
second, and third average return values, a first standard deviation (SDO, a second standard deviation 
(SDz)* and a third standard deviation (SD3), a total number of revisions (revisionsi) occurring more than 
or equal to the first period of time before said evaluating, a total number of revisions (revisions2) 

20 occurring more than or equal to the second period of time before said evaluating, and a total number of 
revisions (revisionss) occurring more than or equal to the third period of time before said evaluating; 

calculating a first degree of freedom (DFO for said first average return value, a second degree 
of freedom (DF2) for said second average return value, and a third degree of freedom (DF3) for said 
third average return value; 

25 determining a first probability (probi) for said first period of time by utilizing said first t-stat 

value and said first degree of freedom, a second probability (prob2) for said second period of time by 
utilizing said second t-stat value and said second degree of freedom, and a third probability (proba) for 
said third period of time by utilizing said third t-stat value and said third degree of freedom; and 
generating a conditional performance score (cond. performance score) for the analyst by 

30 utilizing probi, prob2, and probs,, which may be manipulated to produce said perfonnance score. 

76. A method of evaluating an analyst' s perfonnance, said method comprising: 

a step for utilizing information pertaining to at least one revision made by the analyst involving 
at least one investment, including an upward or downward change in opinion of the analyst with regard 
35 to the at least one investment; 
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a step for calculating a conditional perfonnance score indicative of the analyst's performance 
relative to other analysts, said raw conditional performance score determined at least in part by 
considering a measure of variability of the analyst's performance, an average historical performance of 
the at least one investment following the at least one revision, a standard deviation of the at least one 
5 revision, a number of revisions made by the analyst, and a likelihood that the at least one revision will 
actually produce an expected result; and 

a step for adjusting said conditional performance score according to a return amount adjustment 
to produce a final performance score. 

10 77. A method of evaluating an analyst's performance, said method comprising: 

a step for utilizing information pertaining to at least one upward or downward revision 
including an upward or downward change in opinion of the analyst with regard to the at least one 
investment; 

a step for calculating a perfonnance score indicative of the analyst's perfonnance relative to 
15 other analysts, said perfonnance score determined at least in part by considering a measure of 
variability of the analyst's performance, an average historical perfonnance of the at least one 
investment following the at least one upward or downward revision, a standard deviation of the at least 
one upward or downward revision, a number of upward or downward revisions made by the analyst, 
and a likelihood that the at least one upward or downward revision will actually produce an expected 
20 resxdt; and 

a step for comparing the analyst's performance score against performance scores of other 
analysts to produce at least one of an upward, downward or combined revision ranking. 



wo 01/80124 



PCT/liS01/n691 



1/23 



CONSULTANT DATA 
ON INVESTMENT 

MANAGER 
DATABASE(460+ 
MANAGERS) 



2 



SELECT ALL MANAGERS WfflCH: 
1) HAVE $500+ MILUON IN STOCKS 
2) HAVE BEEN IN BUSINESS 7+ YEARS 
3) HAVE DOMESTIC EQUITIES, 
AND WRITE THEM TOFILE A 



SORT ALL MANAGERS IN FILE ATO 
DESCENDING SEQUENCE ON 
PERFORMANCE. WRITE THE 

SORTED FILE TO FILE B, AND LIST 
CONTENTS OF FILE B. 



T 



CONSULTANT REVIEW OF FILE B. 



I 



PRINT FIRST FIVE NAMES OF 
FILE B, WRITING THEM TO 
FILEC. 



2a 



FILE A 




5a 



FILEC 



FIG. 1 

PRIOR ART 



SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



PCT/USOl/11691 



2/23 




FIG. 2 

PRIOR ART 

SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



PCTAJSOl/11691 




SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



PCTAJSOl/11691 



4/23 



41- 



TECHNICAL 
INPUTS: PRICE 

VOLUME 
INFORMAnON. 
ETC. 




40 



D 

GENERAIl INPUT 
FACTORS FOR EACH 
STOCK (BY TICKER) 



BETA, 

DIVERSMABLE VARIANCE 



50 



1 



BUILD NEURAL 

NETWORK 
FOR EACH STOCK 



60 



1 



STOCK ALPHA PREDICTIONS 



CONSTRUCT & 
OPTIMIZE 
PORTFOUO 



70 



1 



MONITOR 
PORTFOUO 



FIG. 4 

PRIOR ART 



SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



PCTAJSOl/1169! 



5/23 



O 

o 
to 



d 




SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



PCT/US01/1I691 



•6/23 




SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



PCT/USO 1/1 1691 



7/23 




NASDAQ 

IT'S TAX TIME! 

Home I Quotes I Ticker I Newsl Market I Glofc)al I IPOs I Holdings/ 1 Extended I Investorl Persona) 1 Exchange 1 hlasdaq 

Activiy Markets Insiders Trading Tools Finance Traded Funds Store 



3/22/2D01 Maftet Ctosed I N asctaq 1897.70 67.47 * +3.69% I DJIA 3329.48 97.52 ▼ -1.03% I SAP 11 1758 4.56 ▼ -0.41% 

^<plug power 



Rash^Quotes 



SyrrtolLooMJp 



Analyst hfoLAltlX PLUG 
Plug Power, Inc. Nasdaq-NM 



Add a Security 



Compaoy 



News 



Fund- 
amentals 



Chart 



Stodt Analyst 



Ml 



It^jdss 



Report 



RftflUTimfi 



Filings 



Options 



Fiftendftd 



Trading 



Summary 1 Recommendations 1 EPS | Eaminos Surorise ( Forecast I Momentum 
Eaminqs Growth I P/E & Growth Rates i PEG RATIO 



P/E Rafo 



530 



Consensus Reconrunendation 



Detaied Analyst Recommendation 



Sel K I I T I 



Click here tov 



□ □□J] 



Ameritrade® 



A 

Previous Close ^ 

Graph Key 

Price targets are calculated by 
future earnings per share ana then appt^ 
price-to earnings nuitiple, known as 1' 
ratio. 



532- 



/ Momentum (4 Weeks) Quarter End &Rscai Detailed Estimates Submitted 



534 



536 



:> Strong 
Buy 



strong Buy 

Buy 

Hold 

Underperibmi 
SeQ 



ViewHstof 
Analyst Firms 



Each neqpmmendatfan received from contributors is mapped to one of the l/B/E/S| standard 
ratings. A consensus recommendatcn is determined by an average of the numeric values, 
rounoing that mean value to the nearest integer. 



12 Month Price Tafget Range 
Consensus 



Earnings Surprise 



22 



30_ 



33.5 



n/a 



-0.13 
-027 
-0.41 
-0.55 



Q- 



n/a 

-a-DO 



56p99 Dec99 MarOO JunOO SepOO DecOO 

□ Estimale OReported Eamkig 

View eyiincs surprise data table fcy more 

information. 



12/2001 1 of 12 e&t)matB& changed 
Down:0' ' 



> 12/2001 EPS Mean % change 

-1.981% 

•^0191 Z3 1 Montti Ago 



■1.979 C 



□ Ciirent 



1.18 
-1.38. 
-1.58, 
-1.78. 
-1.98. 
-Z18. 
-2.38 



V o 



Estimate momentum measures chanoes in 
analyst sentiment over time and may oe an 
indicator of future price nrovements. 



•136 

— Consensus Estknate 
ARewionUp 



.102 -68 ^ 
#of0ay5 
O Inifial Esttnate Graph Kev 
vReviaonDcwn 



Earnings Growth 



Price/Eamings EPS Summary 



rVa% 



□ 71.25% 



PLUG 

industry 



Estimate Growth is the measure of year on 
year eamings per share (EPS) growth fonn 
me prior fiscal year, expressed as a 
percentage. View Forecast Eamings Growth. 



NE 



□ 2111 



PLUG 

Industry 



Price/Eamings ratio is a widely used stock 
evaluation measure. View PiEdata for the 
next four years for this security.. 



Consensus Eamings Forecasts 



PEG Ratio 



FIG. 5c 



SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



PCTAJSOl/11691 



8/23 



2000A 2001 2002 



Insufficient infonnation to 
the graph tor this symbol 



-1.88 -1.98/12 -1.75/9 

The actual reported earnings per share for 12/2000 
for PLUG was -1 .88. For the fiscal 
year 12/2001 , the consensus mean EPS is -1 .98, 
derived from a total of 12 estimates. 

View annual and quarterly consensus data . 



Home I About Nasdaq i Feedback i Help & Reference 
FqiiHyAlnriftxQptinns | SMIA^, lod&X, SfiaCCb 

Source: Thomson Rnandal l/B/E/S U pdated Daily. 



NASDAQ 



FIG. 5d 



SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



PCTAJSOl/11691 



YES 



FIG. 6 



9/23 



RETRIEVE ANALYST 
FROM DATABASE 



610 




NO 



YES 



CALCULATE SCORE 
FOR ANALYST 



625 




RANK 
ANALYSTS 




620 



DO NOT 
EVALUATE 



SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 PCT/USOl/11691 



10/23 



CALCULME 
UPWARD 
REVISION RETURN 
VALUES 



I 



705 



CALCULATE 
UPWARD 
REVISION T-STAT 
VALUES 



I 



710 



CALCULATE 
UPWARD 
REVISION DEGREES 
OF 
FREEDOM 



I 



715 



LOOK UP UPWARD 
REVISION 
PROBABILTTIES 



I 



720 



CALCULATE 
UPWARD 
REVISION 
CONDITIONAL 
SCORE 



I 



725 



ADJUST SCORE 
ACCORDING TO 
NUMBER OF 
UPWARD REVISIONS 
MODIFIER 
(OPTIONAL) 

i 



730 



ADJUST SCORE 
ACCORDING TO 
lUPWARD REVISIONS 
RETURN AMOUNT 
MODIFIER 
(OPnONAL) 



735 



nG.7 

SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



PCT/USOl/11691 



11/23 



CALCULATE 
DOWNWARD 
REVISION RETURN 
VALUES 



I 



805 



CALCULATE 
DOWNWARD 
REVISION T-STAT 
VALUES 



T 



810 



CALCULAIE 
DOWNWARD 
REVISION DEGREES 
OF FREEDOM 



I 



815 



LOOKUP 
DOWNWARD 
REVISION 
PROBABILFTIES 



T 



820 



CALCULATE 
DOWNWARD 
REVISION 
CONDmONAL SCORE 



I 



825 



ADJUST SCORE 
ACCORDING TO 
NUMBER OF 
DOWNWARD REVISIONS 
MODIFIER 
(OPTIONAL) 



I 



830 



ADJUST SCORE 
ACCORDING TO 
DOWNWARD REVISIONS 
RETURN AMOUNT 
MODIFIER 
(OPTIONAL) 



835 



FIG. 8 

SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



PCT/USOl/11691 



12/23 



^^ ^^ ^ 



910« 



CALCULATE 
UPWARD 
REVISION SHORT 
TERM RETURN 
FACTOR 



T 



905 



CALCULATE 
UPWARD 
REVISION 
INTERMEDIN 

TERM 
RETURN FACTOR 



CALCULATE 
UPWARD 
REVISION LONG 
TERM RETURN 
FACTOR 



915 



CALCULATE 
UPWARD 
REVISION SMALL 
RETURN PENALTY 



920 




no. 9 



SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



PCT/USOI/11691 



13/23 




1010> 



CALCULATE 
DOWNWARD 
REVISION SHORT 
TERM RETURN 
FACTOR 



1005 



CALCULATE 
DOWNWARD 
REVISION 
INTERMEDIATE 

TERM 
RETURN FACTOR 



CALCULATE 
DOWNWARD 
REVISION LONG 
TERM RETURN 
FACTOR 



1015 



CALCULATE 
DOWNWARD 
REVISION SMALL 
RETURN PENALTY 



1020 




1025 



FIG. 10 



SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



PCT/USOl/11691 



• 14/23 



SORT ANALYSTS 
ACCORDING TO 
UPWARD 
REVISION 
SCORE ORDER 



1105 



SEPARATE/DIVIDE 
GROUPS ACCORDING 
TO PREDETERMINED 
CRITERIA 



1110 



ASSIGN SCORE 
TO EACH GROUP 



1115 



FIG. 11 



SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



PCTAJ SOI/1 1691 



• 15/23 



SORT ANALYSTS 
ACCORDING TO 

DOWNWARD 
REVISION 

SCORE ORDER 



1205 



SEPARATE/DIVIDE 
GROUPS ACCORDING 
TO PREDETERMINED 
CRITERIA 



1210 



EACH GROUP 
ASSIGNED SCORE 



1215 



FIG. 12 



SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



PCT/USOl/11691 



140 



176,178 



150 



16/23 






1 

144 

146^-^ 














3DB 






no. 13 




KEYBOARDirMOUSE 



152 



INTERFACE 



\ 




154 



DISPLAY 
INTERFACE 



148 



172 



164 j/'^ 



ROM 



162 



RAM 



178 



COMMUNICATIONS 
PORT 



INFRARED 




INFRARED 


RECEIVER 




TRANSMTITER 


(OPTIONAL) 




(OPTIONAL) 



yl76 V 



174 



FIG. 14 



SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



PCT/USOl/11691 



17/23 




RADIO 




RADIO 


RECEIVER 




TRANSMl'llM 


(OPTIONAL) 




(OPTIONAL) 



170 



FIG. 15 



SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



18/23 



PCT/US01/1I691 




FIG. 16 



SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



PCTAJSOl/11691 




SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



PCT/lfSOl/11691 



■s 

o 

> 



CM 

to 



uo r--> ^ vo oj 

CO x^- r*- ^ oo 

CM v*> o> 

NO VO OO vo 

CO oo so a^ 

oo NO ^ 

oo CO o ^ o 

C^l VO ^ CO 

CM o ^ ^ ^ 

o o o o o 



20/23 



^-i' oo ^ 

P CO ON 

I oo \o 

^ oo r-~ 

^ O oo 

S C3N CO 

o crs « 



I 



^ 3^ s^ 

oo ^ CM o 

^ oo oo vo 

o> >S S 

T— • NO CO \o 

VI s^i ^ 

c4 ^ ^ C5 

o c:? ^ o 

o <=> o o 



> 

CO 



^ oo 

O CO 



CO CO On ^ CO 

CO oo V2 NO 

CM O 0\ oo \0 

CO CO CO oo 

*o CO oo O 

\0 *o oo 

lo CO ^ g oo 

S O S Q O 

C3 o <=> c> 



> 

CO 



c3 
OO 



o 

I— I 
tin 



' — < <— > O CO \o 

^ S ^ T=: 



CM O CO ^ 

ON OO NO 




^ oo ^ CO GO 
-1 ^ X ^ Q O 



s s s s s s 
£9 a a e9 9 a 



a s s s s s s 
a a a .a a a 



SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



PCTAJSOl/11691 



^1 OO On ^ 

JC m r— ^ 

a> <=> »o 

^ ^ lO oo 

So T-^ lO 

, ^ <M ^ 



21/23 



oo cn cn 

On O <^ 

€M OO 
o ^ 

vo so 

oo NO 

m CN ^ 

oo CO — ; 



o o o <=> o 



T3 



oo OS 

^1 <M vo 

^ ""^r o 

^ OS c*^ 

S ^ — CnJ 

O oo O VO 

S O vo 

I ^ S 

I 1— t CD O 



CO 



o o o 



04 so 

<o oo 

04 

SO O 

lo lo 

co oo oo 

CN oo so 
to oo 

o o o 

(O o <=> 



I OS CM 

CN 



^ Os 
> OS lO 



1=: 
o 



o 
o 



CN oo 

cN 
cr> CO 



I 



I 



5 

o 
S 

C3 



m r-* OO 

'.—I to OS 

^ iO 

o< ^ 

oo' OS oo 

C*^ CO 'Tf 

0> w-^ ^ 

CN CO 

O <0 CO 

O <=> O 



CN ^ to 

CO o^ CO 

*0 oo CO 

CN so oo 

CM CM 

Os CO « — » 

OS CO O 

O CO CO 

o c:> 1— • 

o o o 



OS 

oo 
oo 

so 

so 
CN 



CN 
oo 
oo 

OS 

OS 



I 



oo 



a 



a 

I 

o 
6. 



r — CO to 

cs^ ^ CN to CO 

^ OS CO ^ ^ 

so • Os CN 

OS O CO ^ * 

CO so r-* o> oo 

1— • CN CO -rr CO 

^ CO uo CD 

C3 o o o ; 

CZ3 O O O O 



so 
CN 

so 



I 



o ^ ^ oo so OS 

-5 ^ ^ CN oo --^ 

^1 OS ^ r*- 

^1 <M Os T-4 rr ^ 

-JJ C<| ^ CN ''^l- o 

£5 lo CO ; Os ;o 

I CN r*- so CO CO w 

Jid O <0 O 

o o O O <=) o o 

<u c:5 CO o <o o o 

^' ' ' ' ' 

> 

cd 



SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



K:T/US0iyil691 



{ 

o 

s. 



CM ^ CO ^ OS 



22/23 



2 



c 
o 

S 



CM 



CO 0\ 



s 



CL, 

> 

O 



OJ ^ CO OS 



o 

00 



a 

c= 



rS 22 
00 

2 

CO 



c>i tM CO \o r-- 

co 00 a^ ^ 

cn vo 00 10 

\o ♦ o\ CM o 

00 CO 00 CTN *o 

^ ^ CO 

O CS O CP\ <o 

lo CO o c4 

c? ^ 

o CD o c:> 



o 



c 

O 00 
00 



o ^ 



a. 



ON — • 00 

CM vo to On 1/^ 

— • ^ CM ^ 

CM lO CM O 

^-H CO 00 vo O 

10 <M CM 



8 ^ 



?5 8 § 



CZ> c5 CD CD O 



ON 










VO 


ON 


CO 


00 






VO 


^ 




vo 




to 




CM 




so 






00 


CD 




CD 






s 


00 




c=> 


00 


CO 






00 


VD 


ON 


vo 


o\ 




«o 


CM 




CO 


CM 




0 


C3 


CD 


CD 




c5 


0 


<=> 


0 


CD 





CO 



SUBSTITUTE SHEET (RULE 26) 



wo 01/80124 



PCT/USOl/11691 



23/23 











»o 




cn 


vo 




vo 


on 








5^ 








vo 






VO 






S 


m 


vo 




oo 








vo 


»o 


oo 




CO 


VO 


vo 


Ol 






o\ 




VO 


0^ 


^* 








1 








vo 




1 





VO 


Ov 






ON 








vo 












wo 


CTv 


crv 




CO 


vo 


o 


ON 










»— * 


VO 






On 


ON 




O 




CO 




CO 




OO 




VO 


oq 










aS 


CO 










ON 






vo 


vo 





o 

I 

"'I 

■3 
o 
o 

6 

I 

a' 



^ "a 



a 

o ^ 

I 

> 

•s 

a 
o 

a. 



OO 



o 

fx, 



g5 ON 



I 04 g5 



g5 12 



I 



J 



3 



SUBSTITUTE SHEET (RULE 26) 



PATENT COOPERATION TREATY 

PCT 

DECIARAJION OF NON-ESTABLISHMENT Of JNTERNATIONAL SEARCH REPORT 
(PCT Article l7{2Ka), Rules 13ter.1(c) and Ru/e 39) 



Appficarf 3 or agent's file reference 

26119. 120W01 


IMPORTANT DECLARATION 
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1. [Y] The subject matter of the international application relatas to: 
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I. Q mere presentaflor© of information, 
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The subject-matter claimed in claims 1-28,76,77 falls under the 
provisions of Article 17(2) (a){i) and Rule 39.1(i, iii). PCT, such 
subject-matter relating to mathenatical methods and methods of doing 
business. 

Claims 29-75 relate to a conventional system, program product, computer 
readable mediian or electrical signal for performing the business method 
of claims 1-28,76,77. Although these claims do not literally belong to 
the method category, they essentially claim protection for the same 
conmercial effect as the method claims. The International Searching 
Authority considers that searching this subject-matter would serve no 
useful purpose. It is not at present apparent how the subject-matter of 
the present claims may be considered defensible in any subsequent 
examination phase in front of the EPO as Intemationai Preliminary 
Examining Authority with regard to the provisions of Article 33(1) PCT 
(novelty, inventive step); see also Guidelines B-VII, 1-6). 

The applicant's attention is drawn to the fact that claims relating to 
inventions in respect of which no intemationai search report has been 
established need not be the subject of an intemationai preliminary 
examination (Rule 66.1(e) PCT). The applicant is advised that the EPO 
policy when acting as an Intemationai Preliminary Exanining Authority is 
normally not to carry out a preliminary examination on matter which has 
not been searched. This is the case irrespective of whether or not the 
claims are amended following receipt of the search report or during any 
Chapter II procedure. If the application proceeds into the regional phase 
before the EPO, the applicant is reminded that a search may be carried 
out during examination before the EPO (see EPO Guideline C-VI, 8.5), 
should the problems which led to the Article 17(2) declaration be 
overcome. 



